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Information  Requirements  for  Traffic  Awareness  in  a  Free-Flight 
Environment:  An  Application  of  the  FAIT  Analysis 


introduction 

The  present  research  was  undertaken  to  exam  ine  human 
factors  issues  associated  with  National  Airspace  System 
(NAS)  modernization.  The  first  phase  of  this  research 
reviews  the  concept  of  situation  awareness  (SA)  as  it 
relates  to  pilot  surveillance  activities  and  is  presented  in 
a  separate  document  (c£,  Uhlarik  &  Comerford,  2002). 
The  second  phase  of  this  research,  presented  in  the  current 
paper,  identifies  and  classifies  information  requirements 
for  pilot  surveillance  functions  in  the  air  carrier.  Such 
informa?tion  will  be  helpful  in  the  process  of  designing 
new  technologies  for  pilots  in  the  future  NAS. 

1.0  A  REVIEW  OF  RELATED  RESEARCH 

Williams  and  Joseph  (1998)  conducted  a  study  ex¬ 
amining  the  manner  in  which  pilots  mentally  organize 
flight-related  information.  Their  goal  was  to  provide  a 
foundation  for  the  design  of  future  data-link  interfaces 
for  general  aviation  (GA)  pilots.  They  began  by  examin¬ 
ing  the  functional  architecture  of  the  Operational  and 


Supportability  Implementation  System  (OASIS).  This 
system  will  modernize  existing  Automated  Flight  Service 
Station  (AFSS)  equipment  and  provide  GA  pilots  with 
important  weather  and  flight  planning  information.  Ex¬ 
amination  of  the  systems  architecture  identified  48  pieces 
of  data  that  a  GA  aircraft  could  receive  from  and  send  to 
AFSS  via  data  link.  Williams  and  Joseph  presented  GA 
pilots  with  the  list  of  48  items  and  asked  them  to  rate 
each  in  terms  of  its  importance  in  performing  surveil¬ 
lance,  communication,  and  navigation  functions.  For  the 
purposes  of  the  present  paper,  Williams  and  Josephs  most 
relevant  findings  are  represented  in  Figure  1. 

Figure  1  illustrates  the  26  pieces  of  information  that 
GA  pilots  identified  as  being  most  important  in  perform¬ 
ing  surveillance  activities,  and  it  also  represents  pilots’ 
perceived  relations  among  these  items.  Specifically,  the 
number  of  lines  that  connect  two  boxes  is  an  indica¬ 
tion  of  the  perceived  relation  between  the  information 
within  those  boxes.  For  example,  the  perceived  relation 
between  “Rerouting”  and  “Traffic  conflicts”  is  relatively 


Figure  1.  The  typical  conceptual  network  of  data  that  is  important  in  performing  surveillance 
functions  (from  Williams  and  Joseph,  1998). 
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strong,  because  the  nodes  are  connected  directly  by  only 
one  line.  The  relation  between  “Rerouting”  and  “Cruising 
altitude”  is  perceived  as  relatively  weaker,  as  these  nodes  are 
connected  only  indirectly  through  four  other  nodes. 

Although  Williams  and  Josephs  study  was  limited  to 
GA,  the  items  identified  as  important  to  surveillance 
functions  appear  general  enough  that  they  may  also  be 
relevant  to  the  design  of  technologies  for  transport  air 
carriers.  Note  however,  that  the  goal  of  the  present  study 
is  not  limited  to  information  that  can  be  obtained  via  data 
link.  Neverthless,  the  findings  of  Williams  and  Joseph 
(1998)  can  be  combined  with  the  findings  presented  in 
the  following  pages. 

Endsley,  Farley,  Jones,  Midkiff,  and  Hansman  (1998) 
conducted  a  study  that  also  is  relevant  to  the  present 
research.  They  sought  to  identify  the  information  require¬ 
ments  for  commercial  airline  piloting.  To  identify  these 
requirements,  Endsley  et  al.  examined  job  and  taskanalyses 
that  were  performed  in  the  past  and  conducted  interviews 
with  active  airline  pilots.  There  were  three  outcomes  of 
their  study:  a  goal  hierarchy,  a  goal-directed  task  analysis, 
and  alist  of  information  requirements.  The  goal  hierarchy 
identifies  a  pilot  s  “basic  goal”  as  “Get  aircraft  from  origin 
to  destination  safely,  legally,  with  satisfactory  levels  of 
comfort  and  service  to  passengers,  on  schedule,  and  in 
an  efficient  manner.”  Four  sub-goals  were  identified  as 
being  important  in  achieving  the  basic  goal:  “Select  best 
path  to  destination,”  “Execute  desired  flight  path  safely, 
efficiently,  and  ride  with  comfort,”  “Manage  resources 
effectively,”  and  “Satisfy  the  customer.”  For  each  of  the 
four  goals,  the  necessary  general  tasks  were  identified, 
and  for  each  general  task,  the  necessary  sub-tasks  were 
identified.  For  example,  the  general  task,  “Assess  flight 
plan”  appears  under  the  goal  “Select  best  path  to  destina¬ 
tion,”  and  the  sub-task  “Push  back  from  gate”  appears 
under  the  general  task  “Assess  flight  plan.”  To  perform 
th  e  goal-directed  task  analysis ,  Endsley  et  al.  used  the  goal 
hierarchy  as  a  template  and  inserted  the  pilot  informa¬ 
tion  requirements  under  the  appropriate  headings.  To 
produce  the  final  list  of  information  requirements ,  the 
goal-directed  task  analysis  was  used  to  extract  informa¬ 
tion  requirements  that  were  common  across  goals,  and 
this  list  was  divided  into  several  categories  (e.g.,  aircraft 
state,  airports,  traffic). 

The  aim  of  the  present  research  may  appear  redun¬ 
dant  with  the  work  of  Endsley  et  al.  (1998),  in  the  sense 
that  the  questions  posed  in  the  present  research  can  be 
answered  by  simply  examining  the  relevant  categories 
of  information  requirements  identified  by  Endsley  et  al. 
In  fact,  several  of  the  categories  are  directly  relevant  to 
the  present  research.  For  example,  the  list  of  informa¬ 
tion  requirements  contains  categories  such  as  “Traffic,” 


“Terrain/ obstacles,”  and  “Weather.”  However,  there  are  at 
least  two  limitations  of  Endsley’s  work  that  do  not  apply 
to  the  present  research.  First,  their  analysis  performed  by 
Endsley  et  al.  was  intended  to  be  “as  technology- free  as 
possible”  (p.  4).  While  a  technology- free  analysis  is  desir¬ 
able  for  purposes  of  generalization,  it  may  be  unrealistic 
in  the  context  of  a  technology-laden  environment  like 
the  cockpit  (and  the  free-flight  environment,  for  that 
matter).  Second,  Endsley  et  al.  purposefully  ignored 
information  that  is  static.  For  example,  they  did  not 
include  information  related  to  procedures  or  rules.  The 
present  research  does  not  attempt  to  make  a  distinction 
between  static  and  dynamic  information,  insofar  as  this 
distinction  can  become  quite  unclear. 

The  problem  with  the  static/dynamic  distinction  is 
exemplified  by  the  rationale  that  Endsley  et  al.  marshaled 
in  their  argument  against  including  static  information. 
Specifically,  Endsley  et  al.  implied  that  a  rule,  by  its  very 
nature,  is  static.  However,  rules  often  have  dynamic  infor¬ 
mation  embedded  within  them  (e.g.,  “If  I’m  flying  above 
X  feet,  I  may  fly  at  my  cruise  speed  of  Y  knots.  However, 
if  I’m  flying  below  X  feet,  I  must  fly  at  Z  knots”).  In 
other  words,  it  can  be  argued  that  rules  are  dynamic  in 
the  sense  that  the  appropriate  rule  changes,  depending 
on  the  surrounding  (dynamic)  circumstances. 

An  even  more  important  reason  for  ignoring  the  distinc¬ 
tion  between  static  and  dynamic  information  is  related 
to  the  two  primary  goals  for  identifying  information 
requirements:  (1)  the  requirements  might  be  used  as  a 
tool  in  developing  a  training  program,  and/or  (2)  they 
may  be  used  as  a  tool  in  the  development/enhancement 
ofa  user  interface.  For  both  of  these  applications,  static 
information  is  just  as  important  as  dynamic  informa¬ 
tion.  One  certainly  would  not  want  to  develop  a  training 
program  that  ignores  static  information  (e.g.,  rules),  nor 
would  one  want  engineers  developing  a  piece  of  technol¬ 
ogy  that  fails  to  present  the  pilot  with  static  information 
(e.g.,  a  navigation  tool  that  fails  to  present  information 
regarding  the  location  of  unchanging  fixes). 

2.1  Summary  of  Related  Research 

The  goal  of  the  present  paper  is  to  identify  and  classify 
information  requirements  for  pilot  surveillance  activities 
in  the  air  carrier.  The  previously  summarized  research 
projects  do  not  fully  satisfy  this  goal.  Although  the 
research  performed  by  Williams  and  Joseph  is  relevant, » 
they  examined  only  information  that  might  be  obtained 
via  data  link  and  concentrated  on  GA.  Further,  the  re¬ 
search  performed  by  Endsley  et  al.  (1998)  was  limited  in 
the  sense  that  technology  was  not  considered,  and  only 
dynamic  aspects  of  the  task  were  considered. 
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3.0  THE  CURRENT  RESEARCH! 
BACKGROUND  INFORMATION 

3.1  The  Goal  of  the  Current  Research 

The  research  performed  by  Endsley  et  al,  (1998)  dem¬ 
onstrates  how  complex  and  detailed  a  task  analysis  of  all 
surveillance  activities  would  be.  The  research  described 
below  is  restricted  to  an  analysis  of  the  pilots  task  of 
monitoring  traffic  during  the  cruise  segment  of  flight. 

It  is  further  limited  to  a  free-flight  environment,  which 
has  been  proposed  as  the  operational  concept  for  the 
future  NAS  (RTCA,  1995).  Therefore,  the  goal  of  the 
present  research  is  to  identify  the  information  required 
for  the  air  carrier  pilot  to  achieve  “traffic  awareness55  in 
the  free-flight  environment. 

Traffic  awareness  is  one  component  of  the  more  general 
construct  of  situation  awareness.  As  discussed  in  aseparate 
document  (Uhlarik  &  Comerford,  2002) ,  there  are  many 
definitions  of  situation  awareness.  According  to  Wickens 
(1995),  situation  awareness  is  the  “continuous  extraction 
of  environmental  information  about  a  system  or  environ¬ 
ment,  the  integration  of  this  information  with  previous 
knowledge  to  form  acoherent  mental  picture,  and  the  use 
of  that  picture  in  directing  further  perception,  anticipat¬ 
ing,  and  responding  to  future  events”  (p.  1).  Although 
research  has  been  conducted  on  several  components  of 
SA  that  are  related  to  traffic  awareness  (e.g.,  environment, 
spatial,  temporal,  and  navigation  awareness),  research  on 
traffic  awareness  has  been  sparse.  As  a  result,  no  formal 
definition  of  traffic  awareness  exists.  The  following  defini¬ 
tion  of  traffic  awareness  is  used  for  the  present  research: 
Traffic  awareness  is  having  knowledge  of  the  information 
necessary  to  obtain  and  maintain  self-separation  in  the 
free-flight  environment,  where  successful  self-separation  is 
defined  as  keeping  ownship  separated  from  other  aircraft 
by  5  miles  horizontally  or  1,000  feet  vertically. 

While  the  pilot  in  the  free-flight  environment  will 
obtain  traffic  information  from  several  sources  (e.g.,  out- 
the-window  information,  radio  communications),  the 
primary  source  of  information  presumably  will  be  the 
Cockpit  Display  of  Traffic  Information  (CDTI).  Gener¬ 
ally  speaking,  a  CDTI  is  a  cockpit  display  that  provides 
the  pilot  with  traffic  information  (e.g.,  ownship  posi¬ 
tion,  the  position  of  other  aircraft),1  Because  the  CDTI 
will  be  the  pilot  s  primary  source  of  traffic  information, 
identifying  the  information  required  for  traffic  awareness 
will  be  helpful  in  the  design  and  certification  of  a  CDTI 
for  the  free-flight  environment. 

The  Function  Allocation  Issues  and  Tradeoffs  (FAIT) 
analysis  (c£,  Riley,  1993)  was  chosen  to  identify  the 
relevant  information  requirements.  In  general  terms. 


FAIT  analysis  is  a  systematic  procedure  for  identifying 
human  factors  issues  in  human-machine  systems.  The 
FAIT  analysis  was  chosen  because  it  appears  to  have  great 
potential,  and  it  provides  substantially  more  informa¬ 
tion  than  a  traditional  task  analysis.  Among  other  things, 
the  FAIT  analysis  (1)  assists  in  identifying  information 
requirements,  (2)  yields  numerical  values  that  represent 
the  relative  influence  and  the  relative  sensitivity  of  im¬ 
portant  aspects  within  the  system,  and  (3)  allows  for  the 
examination  of  tradeoffs  in  a  human-machine  system 
(e.g.,  when  aspect  X  of  the  system  reaches  a  desirable 
state,  aspect  Y  consequently  may  reach  an  undesirable 
state).  In  addition,  the  FAIT  analysis  places  equal  em¬ 
phasis  on  the  human  and  machine  components  of  the 
system,  thereby,  allowing  the  analyst  to  include  purely 
psychological  constructs  (e.g.,  mental  workload)  of  the 
human  system  component. 

Despite  its  potential  utility,  the  FAIT  analysis  has  been 
used  infrequently,  and  most  published  studies  that  utilize 
it  have  been  conducted  by  its  developer  (i.e.,  Riley,  1989; 
Riley,  1992;  Riley,  Lyall,  &  Wiener,  1993).  Therefore,  in 
addition  to  identifying  information  requirements,  die 
present  research  also  provides  a  vehicle  for  evaluating 
the  utility  of  the  FAIT  analysis.  Specifically,  to  assess 
the  utility  of  the  FAIT  analysis,  the  information  require¬ 
ments  obtained  via  the  FAIT  analysis  are  compared  with 
the  information  requirements  identified  by  Endsley  et 
al.  (1998). 

3.2  A  Definition  of  die  Human-Machine  System 

The  FAIT  analysis  was  developed  with  the  assump¬ 
tion  that  it  would  be  used  for  the  analysis  of  complex 
human-machine  systems  (Riley,  1993)  .  Toward  this  end, 
a  necessary  first  step  is  to  identify  components  of  the  hu¬ 
man-machine  system  before  the  FAIT  analysis  is  applied. 
For  example,  because  complex  human-machine  systems 
generally  include  at  least  some  level  of  automation,  the 
analysis  requires  the  researcher  to  determine  the  amount 
of  control  the  machine  possesses,  and  the  following  com¬ 
ponents  must  be  identified:  the  human,  the  environment, 
and  the  machine.  Obviously,  the  human  component  of 
the  system  to  be  analyzed  in  the  current  research  is  the 
pilot  (although  co-pilots,  pilots  of  other  aircraft,  and 
air  traffic  controllers  are  ignored  at  this  stage,  they  are 
taken  into  account  later  in  the  analysis).  The  environ¬ 
ment  for  the  human-machine  system  is  identified  as  a 
modernized  NAS,  and  therefore,  the  current  research 
assumes  that  aircraft  are  equipped  with  Automatic  De¬ 
pendent  Surveillance-Broadcast  (ADS-B),  which  will 
allow  aircraft  to  transmit  and  receive  three-dimensional 
position  information  via  data  link.  With  such  informa- 


1  (A  more  detailed  description  of  a  particular  example  of  a  CDTI  is  provided  later  in  this  document.) 
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tion,  the  pilot  may  be  responsible  for  self-separation  in 
most  instances,  but  the  air  traffic  manager  (ATM)  will 
monitor  traffic  and  control  by  exception.  In  other  words, 
the  pilot  will  perform  tasks  necessary  for  self-separation, 
and  the  ATM  will  intervene  only  in  cases  where  alert 
zones  overlap.  For  the  current  purposes,  an  alert  zone 
is  defined  as  a  spherical  area  that  surrounds  an  aircraft 
by  five  nautical  miles  on  the  horizontal  dimension  and 
1,000  feet  on  the  vertical  dimension.  However,  if  tech- 
nology  permits,  these  alert  zones  may  be  smaller.  A  final 
assumption  was  that  the  pilot  would  access  the  ADS-B 
information  using  a  CDTI,  which  would  be  integrated 
into  the  current  aircraft  avionics  suite.  The  CDTI  was 
identified  as  the  machine  component  of  the  system  for 
the  application  of  the  FAIT  analysis. 


3.3  NASA-AMES  CDTI 

Because  the  FAIT  analysis  is  intended  for  use  in  the 
early  stages  of  the  design  process  (Riley,  1993),  a  search 
was  conducted  for  a  candidate  CDTI  sill  in  the  early  stages 
of  design.  In  fact,  researchers  at  National  Aeronautics  and 
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rather  than  a  specific  display  format  or  configuration  of 
their  prototype  CDTI.  The  configuration  and  capabili¬ 
ties  of  the  NASA-AMES  CDTI  varies  considerably  (e.g., 
Cashion, Mackintosh,  McGann,  &  Lozito  1 997;  Johnson 
etal.,  1997;  Mackintosh  etal.,  1998),  and  the  display  used 
in  the  current  analysis  consists  of  a  somewhat  arbitrary 
collection  of  NASA  display  concepts.  A  brief  description 
of  this  display  appears  below,  but  Appendix  A  provides 
a  more  thorough  description  of  the  CDTI. 

Figure  2  presents  a  schematic  of  the  NASA-AMES 
CDTI,  which  allows  the  pilot  to  view  ownship  position 
(depicted  as  a  white  chevron  in  the  lower  center  portion  of 


Figure  2)  in  relation  to  other  aircraft.  In  addition,  altitude 
of  other  aircraft  can  be  assessed  in  relation  to  ownship 
(Le.,  other  aircraft  are  color-coded  according  to  altitude 
and  their  datatags  display  altitude  numerically). 

Using  the  toolbar  at  the  bottom  of  the  display,  the  pilot 
can  configure  the  CDTI  with  there  viewing  options:  (1) 
altitude  can  be  displayed  in  absolute  or  relative  terms,  (2) 
datatags  can  be  viewed  or  hidden,  and  (3)  the  predicted 
routes  of  aircraft  can  be  viewed  or  hidden.  In  addition, 
when  the  appropriate  option  is  chosen,  the  pilot  can:  (1) 
request  that  datatags  be  rearranged  for  optimum  view¬ 
ing,  (2)  change  the  amount  of  time  the  predictor  lines 
represent,  and  (3)  view  the  predictor  lines  in  one  of  two 
formats.  Finally,  the  Route  AssessmentTool  (RAT)  allows 
the  pilot  to  assess  and  accept  (If  so  desired)  changes  in 
altitude,  vertical  speed,  and  heading.  Therefore,  a  pilot 
who  is  considering  changes  (e.g.,  in  altitude)  is  allowed 
to  determine  if  an  alert  zone  contact  is  likely  before  the 
changes  are  initiated. 

4.0  A  STEP  BY-STEP  DESCRIPTION  OF 
THE  FAIT  ANALYSIS 

This  summrizes  of  the  current  findings  and,  in 
the  context  of  the  current  findings,  presents  a  brief 
overview  of  the  FAIT  analysis  procedures.  A  complete 
description  of  the  analysis  appears  in  the  Function 
Allocation  Issues  and  Tradeoffs;  User's  Manual  (Riley, 
1993).  The  FAIT  analysis  Is  divided  into  six  major 
steps  for  ease  of  understanding. 

4.1  Step  li  Develop  a  Model  of  Information  Flow 

The  goal  In  the  first  step  is  to  determine  the  model 
of  information  flow  within  the  system.  There  are  three 
tasks  to  be  completed  in  determining  the  appropriate 
model.  The  first  task  is  to  identify  the  machines  level  of 
autonomy.  The  FAIT  users  manual  provides  a  descrip¬ 
tion  of  12  levels  of  autonomy  (c£,  Riley,  1993,  p.  6-8). 
The  highest  level  represents  a  machine  with  complete 
autonomy  (i.e.,  the  machine  performs  control  actions 
without  informing  or  interacting  with  the  operator),  and 
the  lowest  level  represents  a  machine  with  no  autonomy. 
Utilizing  complex  algorithms  (c£,  Johnson  et  al.,  1997), 
the  NASA-AMES  CDTI  attempts  to  identify  other 
aircraft  that  are  likely  to  have  alert  zone  contact  with 
ownship,  and  therefore,  the  current  analysis  placed  the 
NASA  CDTI  at  the  “Simple  aid”  level  of  autonomy.  This 
classification  fits  nicely  with  Rileys  (1993)  example  of  a 
target  recognizer  that  simply  attempts  to  categorize  radar 
returns  as  belonging  to  targets,  and  as  such  was  classified 
as  a  “Simple  aid.” 

The  second  task  In  Step  1  is  to  identify  the  machines 
level  of  intelligence.  Like  the  first  task,  the  researcher  is 


to  choose  the  machine  s  level  of  intelligence  from  a  list 
of  seven  levels  (c£,  Riley,  1993,  p.  8-9).  The  highest  level 
represents  a  machine  with  a  high  degree  of  “intelligence” 
(i.e. ,  the  machine  has  information  regarding  the  operator  s 
physical  state  and  intent  and  therefore  can  predict  the 
operators  behavior),  and  the  lowest  level  represents  a 
machine  with  virtually  no  intelligence  (i.e.,  it  presents 
only  raw  data). 

Within  this  category,  a  machine  viewed  as  “Person¬ 
alized”  is  one  that  can  be  configured  according  to  the 
operators  preferences.  Therefore,  because  some  aspects 
of  the  NASA  CDTI  can  be  manipulated  in  accordance 
with  the  pilot  s  preferences  (e.g.,  datatags  may  be  shown 
or  hidden,  predictors  maybe  shown  or  hidden,  the  length 
of  the  predictors  may  be  varied  according  to  taste),  the 
current  analysis  placed  the  NASA  CDTI  in  the  “Person¬ 
alized”  category. 

After  the  level  of  autonomy  and  level  of  intelligence 
have  been  identified,  the  third  and  final  task  in  this  step 
Is  aimed  at  creating  an  Information  flow  model  for  the 
current  system.  The  general  model  of  information  flow 
(Figure  3),  serves  as  the  foundation  for  all  resultant  mod¬ 
els  of  Information  flow  when  using  the  FAIT  analysis. 
The  FAIT  users  manual  provides  the  researcher  with 
12  templates  that  correspond  to  the  levels  of  autonomy 
and  seven  templates  that  correspond  to  the  levels  of 
intelligence.  Generally  speaking,  the  level  of  autonomy 
determines  which  nodes  are  removed  from  the  “machine 
output”  quadrant  of  the  general  model,  because  they  are 
not  relevant.  Similarly,  the  level  of  intelligence  determines 
which  nodes  are  removed  from  the  “machine  input”  quad¬ 
rant,  because  they  are  not  relevant.  The  two  templates 
that  result  from  the  current  level  of  autonomy  (simple 
aid)  and  the  current  level  of  intelligence  (personalized) 
are  fused  to  produce  a  more  specific  model  of  Informa¬ 
tion  flow  for  the  current  analysis.  This  model  is  shown 
in  Figure  4. 

4.2  Step  2%  Decompose  the  Model  Into 
Characteristics 

The  model  of  Information  flow  that  is  created  in  the 
first  step  Is  used  as  a  tool  in  the  second  step.  Specifically, 
for  each  node  in  the  model,  important  characteristics  are 
listed.  Although  Riley  (1993)  does  not  provide  a  formal 
definition  of  a  characteristic,  here  it  is  considered  to  be 
an  important  aspect  of  the  system  that  can  vary  from  a 
desirable  to  an  undesirable  state.  Characteristics  in  desir¬ 
able  states  are  important  to  a  system  s  proper  functioning. 
Characteristics  in  undesirable  states  are  equally  Important 
because  they  may  lead  to  malfunctions  In  the  system.  For 
example,  Riley  suggests  that,  in  a  typical  system,  charac¬ 
teristics  of  the  “Plan  own  action”  node  might  be  “Level 
of  mental  workload”  and  “Crew  coordination.” 
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Although  Riley  (1993)  includes  numerous  examples, 
the  analyst  is  expected  to  determine  the  relevant  char- 
acteristics  for  the  nodes  in  the  system  of  interest.  In 
determining  these  characteristics,  Riley  states  that  you 
should  consider  the  characteristics  of  the  system  under 
investigation,  as  well  as  all  characteristics  of  the  operational 
environment  and  all  related  systems  that  may  influence 
or  be  influenced  by  how  the  particular  system  works. 
Therefore,  the  current  analysis  includes  more  than  the 
pilot,  the  traffic  environment,  and  the  CDTI  that  were 
identified  at  the  onset  of  the  analysis.  At  this  point,  the 
analysis  also  includes  such  things  as  the  flight  manage¬ 
ment  system,  air  traffic  controllers,  fellow  crew  members, 
pilots  of  other  aircraft,  etc. 

All  nodes  from  the  general  model  of  information  flow 
and  the  outcome  for  Step  Two  of  the  current  analysis  are 
included  in  Appendix  B.  In  addition,  Rileys  definition 
(1993)  of  each  node  is  provided.  Nodes  not  included  in 
the  current  model  appear  in  gray,  and  justifications  are 
made  for  their  exclusion.  Appendix  B  also  presents  a 
question  for  each  node.  These  questions  were  formulated 
for  the  current  analysis  to  assist  in  the  identification  of 
relevant  characteristics.  Finally,  Appendix  B  presents  the 
important  characteristics  that  were  identified  for  each 
node  in  the  system. 

A  preliminary  analysis  yielded  116  characteristics. 
However,  numerous  characteristics  yielded  redundant 
information  in  later  steps  of  the  analysis.  For  example, 
the  final  analysis  uses  one  characteristic,  “Ownship  state,” 
to  represent  six  characteristics  that  yielded  redundant 
information  in  the  preliminary  analysis  (i.e.,  “Ownship 
position,”  “Ownship  speed,”  “Ownship  heading,”  “Own- 
ship  vertical  speed,”  “Ownship  altitude,”  and  “Ownship 
attitude”).  The  final  analysis  yielded  a  total  of  68  unique 
characteristics.  A  characteristic  was  only  counted  once, 
even  if  it  was  associated  with  more  than  one  node. 

4.3  Step  3;  Create  a  Matrix  of  the  Characteristics 
and  Their  Interactions 

The  third  step  requires  placement  of  characteristics  in 
a  matrix  (illustrated  in  Table  1)  that  is  created  by  placing 
a  list  of  all  characteristics  along  the  left  margin  and  the 
same  list  along  the  top  of  the  matrix.  Although  the  lists 
are  identical,  the  characteristics  along  the  left  margin  are 
referred  to  as  the  “drivers,”  whereas  the  characteristics 
along  the  top  are  referred  to  as  the  “receivers.”  These 
labels  represent  the  manner  in  which  questions  are  posed 
regarding  the  relation  among  characteristics. 

The  matrix  is  used  to  determine  the  relations  among 
characteristics.  Beginning  with  the  first  driver,  the  re¬ 
searcher  asks  two  questions  for  each  receiver:  ( 1 )  “Is  there 
any  situation  in  which  the  driver  characteristic  influences 


the  receiver  characteristic?”  and  (2)  “Is  there  any  situa¬ 
tion  in  which  some  limitation  of  the  driver  characteristic 
places  a  requirement  on  the  receiver  characteristic?”  If 
the  answer  is  “yes”  to  either  of  these  questions,  an  entry 
is  placed  in  the  matrix  cell  where  the  driver  and  receiver 
intersect.  Once  every  receiver  has  been  considered  in 
relation  to  the  first  driver,  this  process  is  repeated  for  the 
remaining  drivers  in  the  list. 

Appendix  C  contains  the  matrix  produced  for  the 
current  analysis.  Matrix  cells  that  are  shaded  dark  gray 
represent  the  negative  diagonal  of  the  matrix.  These  cells 
represent  points  where  characteristics  intersect  with  them¬ 
selves.  The  lighter  shade  of  gray  and  bold  n  umbering  used 
in  other  matrix  cells  will  be  explained  in  Step  5. 

4.4  Step  4:  Obtain  Rough  Estimates  of  the 
Relative  Importance  of  Characteristics 

The  fourth  step  in  the  FAIT  analysis  identifies  the 
relative  importance  of  the  characteristics  by  summing  the 
rows  and  columns  of  the  matrix.  Row  totals  and  ranks 
represent  “influence,”  and  a  characteristic  that  is  highly 
influential  is  one  that  affects  many  other  characteristics  of 
the  system.  Column  totals  and  ranks  represent  “sensitiv¬ 
ity,”  and  a  characteristic  that  is  highly  sensitive  is  one  that 
is  vulnerable  to  the  effects  of  many  other  characteristics 
of  the  system. 

Appendix  D  provides  a  convenient  format  for  exam¬ 
ining  the  influence  and  sensitivity  scores  for  the  current 
system.  Table  D1  presents  the  68  characteristics,  their 
respective  influence  scores,  and  their  relative  rank.  The 
characteristics  are  presented  in  an  order  consistent  with 
their  relative  rank  on  the  influence  dimension.  Similarly, 
Table  D2  presents  the  68  characteristics,  their  respective 
sensitivity  scores,  and  their  relative  rank.  They  also  are 
presented  in  an  order  consistent  with  their  relative  rank 
on  the  sensitivity  dimension. 

Although  there  are  no  formal  criteria  for  identifying 
the  most  important  characteristics,  Riley  (personal  com¬ 
munication,  June  1,  1999)  suggests  concentrating  on 
characteristics  that  score  the  highest  on  influence  and 
sensitivity.  Hence,  the  relative  frequency  associated  with 
the  90th  percentile  point  was  identified  for  the  distributions 
of  influence  and  sensitivity  scores.  This  arbitrary  cutoff 
created  a  more  manageable  list  of  characteristics.  Seven 
scores  fell  above  the  90th  percentile  for  both  distributions, 
and  they  are  denoted  by  a  gray  star  in  Appendix  D. 

The  following  characteristics  were  identified  as  hav¬ 
ing  the  most  influence  in  the  system  and  are  presented 
in  rank  order: 

1)  Weather 

2)  General  piloting  skills 

3)  Time  of  day 
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Figure  3,  The  general  model  of  information  flow  (taken  from  Riley,  1993), 
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Figure  4.  The  model  of  information  flow  in  the  current  system  (simple  aid/ personalized)  system. 


4)  Terrain 

5)  Ownship  state 

6)  Level  of  pilot  mental  workload 

7)  Perceived  time  pressure 

The  following  characteristics  were  identified  as  being 
the  most  sensitive  (or  vulnerable)  and  are  also  presented 
in  rank  order: 

1)  Type  of  action  chosen  by  the  pilot 

2)  Level  of  pilot  mental  workload 

3)  Appropriateness  of  planned  action 

4)  Ownship  state 

5)  Air  traffic  manger’s  amount  of  mental  workload 

6)  Accuracy  of  current  machine  model 

7)  Level  of  confidence  in  planned  action 

Two  characteristics  (“Ownship  state”  and  “Level  of  pilot 
mental  workload”)  were  identified  as  both  very  influential 
and  highly  sensitive.  Not  surprisingly,  the  state  of  the 
aircraft  affects  many  aspects  of  the  system,  and  at  the 
same  time,  is  affected  by  many  aspects  of  the  system.  The 
same  may  be  said  of  pilot  mental  workload.  Given  they 
fall  above  the  90th  percentile  point  on  both  dimensions, 
these  two  characteristics  are  probably  important. 

Because  traffic  awareness  is  so  important  in  the  free- 
flight  environment,  two  specific  characteristics  (viz,,  “Ac¬ 
curacy  of  the  current  machine  model”  and  “Accuracy  of 
pilot  s  current  world  model”)  were  combined.  These  two 
characteristics  are  denoted  by  black  stars  in  Appendix  D, 
These  two  characteristics,  taken  together,  represent  the 
more  global  characteristic  of  “Traffic  awareness.”  The 
analysis  suggests  that  traffic  awareness  is  no  more  influ¬ 
ential  than  the  higher  ranking  of  the  two  characteristics 
of  which  it  is  composed  (i.e,,  “Accuracy  of  pilot  s  current 


world  model”  and  “Accuracy  of  current  machine  model”). 
However,  traffic  awareness  is  a  more  sensitive  characteristic 
than  when  its  components  are  examined  separately.  When 
“Accuracy  of  the  current  machine  model”  and  “Accuracy 
of  pilots  current  world  model”  are  removed  from  the 
analysis  and  are  replaced  with  the  more  global  charac¬ 
teristic  of  “Traffic  awareness,”  the  list  of  most  sensitive 
characteristics  appears  as  follows: 

1)  Type  of  action  chosen  by  the  pilot 

2)  Level  of  pilot  mental  workload 

3)  Traffic  awareness 

4)  Appropriateness  of  planned  action 

5)  Ownship  state 

6)  Air  traffic  manger  s  amount  of  mental  workload 

7)  Level  of  confidence  in  planned  action 

Despite  having  relatively  little  influence ,  the  sensitiv¬ 
ity  of  traffic  awareness  suggests  that  it  is  a  vulnerable 
characteristic. 

In  summary,  Step  4  was  used  to  Identify  the  most 
important  characteristics  in  the  system.  The  seven  most 
influential  characteristics  and  seven  most  sensitive 
characteristics  were  extracted  from  the  list  of  68  char¬ 
acteristics.  Two  characteristics  (ownship  state  and  level 
of  pilot  mental  workload)  were  identified  as  being  both 
highly  influential  and  highly  sensitive.  Therefore,  these 
characteristics  have  been  identified  as  especially  important 
characteristics.  Finally,  two  components,  “Accuracy  of  the 
current  machine  model”  and  “Accuracy  of  pilots  current 
world  model,”  were  combined  to  create  the  characteristic 
called  “Traffic  awareness.”  Traffic  awareness  was  no  more 
influential 'than  the  higher  ranking  of  its  two  components. 
However,  traffic  awareness  was  one  of  the  most  sensitive 
characteristics  of  the  system. 


Table  1.  Structure  of  a  matrix  used  in  the 
FAIT  analysis. 


Receivers 


Characteristic  1 

Characteristic  2 

Characteristic  3 

Characteristic  1 

Characteristic  2 

Characteristic  3 

Table  2.  A  sample  matrix  illustrating  the 
manner  in  which  tradeoffs  are  identified. 


Receivers 


Characteristic  1 

Characteristic  2 

Characteristic  3 

Characteristic  1 
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Characteristic  2 
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Characteristic  3 
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4.1  Step  5:  Identify  Tradeoffs  in  the  System 

The  fifth  step  in  the  FAIT  analysis  identifies  potential 
tradeoffs,  which  are  found  by  identifying  symmetrical 
relations  within  the  matrix.  Symmetrical  relations  are 
identified  by  folding  the  matrix  along  the  negative  di¬ 
agonal  and  identifying  entries  that  are  superimposed  on 
each  other.  These  entries  represent  potential  tradeoffs 
between  characteristics.  For  example,  there  is  one  sym¬ 
metrica]  relation  in  Table  2,  and  it  is  identified  by  the 
two  lightly  shaded  cells  with  the  bold  number  “1”  in 
them.  The  potential  for  a  tradeoff  exists  in  this  case, 
because  characteristic  1  influences  characteristic  2  and 
characteristic  2  influences  characteristic  1 . 

The  coding  scheme  used  in  Table  2  also  is  used  in  Ap¬ 
pendix  C,  where  lightly  shaded  cells  that  contain  bolded 
number  “  1  ”s  identify  symmetries.  Because  symmetries 
in  Appendix  C  may  be  difficult  to  visualize,  Table  D3  of 
Appendix  D  presents  a  list  of  the  30  characteristics  that 
have  been  identified  as  having  the  potential  for  a  tradeoff 
with  another  characteristic.  Note  that  Table  D3  consists 
of  124  lines,  but  this  number  is  misleading.  For  example, 
“Level  of  pilot  mental  workload”  is  listed  under  “Ownship 
state,”  and  “Ownship  state”  is  listed  under  “Level  of  pilot 
mental  workload.”  However,  these  two  lines  represent 
only  one  potential  tradeoff.  Therefore,  the  current  analysis 
identified  a  total  of  62  potential  tradeoffs. 

Because  there  are  62  potential  tradeoffs,  the  charac¬ 
teristics  that  were  identified  in  Step  Four  were  used  to 
shorten  this  list  (Riley,  personal  communication,  June 
1,  1999),  and  the  resulting,  abbreviated  matrix  appears 
in  Table  D4  of  Appendix  D.  This  abbreviated  matrix 
includes  only  the  characteristics  that  are  highly  sensitive, 
highly  influential,  and  are  components  of  traffic  awareness. 
Therefore,  the  matrix  includes  1 3  characteristics.  To  make 
the  symmetrical  relations  clear,  the  tradeoffs  are  coded 
with  numbers  in  Table  D4.  For  example,  “Accuracy  of 
pilot  s  current  world  model”  influences  “Ownship  state” 
and  vice  versa.  Therefore,  there  are  two  boxes  in  Table 
D4  that  are  assigned  the  number  “1.”  The  number  as¬ 
signed  to  a  pair  is  used  only  to  distinguish  one  pair  from 
another. 

As  seen  in  Table  D4,  1 1  potential  tradeoffs  require 
examination.  Riley  (1993)  suggests  that,  in  general,  the 
most  useful  product  of  the  FAIT  is  the  documentation 
of  possible  scenarios.  Potential  tradeoffs  are  examined  by 
creating  a  tradeoff  scenario  for  each  pair  of  characteristics 
that  produce  symmetry  in  the  matrix.  The  following  list 
provides  scenarios  that  would  exemplify  the  tradeoff  as¬ 
sociated  with  each  pair  of  characteristics.  Each  scenario 
describes  a  set  of  possible  circumstances  that  illustrates 
the  potential  tradeoff.  If  scenarios  cannot  be  provided, 
then  the  tradeoff  is  probably  nonexistent.  However,  if  a 
reasonable  scenario  can  be  generated,  then  the  tradeoff 


should  be  taken  seriously.  In  other  words,  this  task  de¬ 
termines  which  of  the  potential  tradeoffs  (identified  in 
the  matrix)  are  true  tradeoffs.  Tradeoffs  are  examined  by 
considering  both  characteristics  in  a  pair  (e.g.,  “Ownship 
state”/  “Accuracy  of  pilots  current  world  model”)  as  a 
driver  and  a  receiver.  Each  characteristic  of  the  system 
can  presumably  be  in  various  states  ranging  from  opti¬ 
mal,  to  sub-optimal,  to  its  poorest  state.  In  other  words, 
the  state  of  each  characteristic  can  range  in  terms  of  its 
desirability.  For  example,  the  accuracy  of  a  pilots  world 
model  can  range  from  desirable  (i.e.,  it  might  be  perfect) 
to  undesirable  (i.e.,  the  pilot  currently  may  not  have  a 
mental  model  of  the  world  situation).  To  represent  a  true 
tradeoff,  the  characteristics  in  a  pair  must  be  inversely 
related  in  terms  of  desirability;  that  is,  an  increase  in  the 
desirability  of  Characteristic  A  must  lead  to  a  decrease  in 
the  desirability  of  Characteristic  B  and  vice  versa.  Some 
pairs  may  yield  direct  relations  (i.e.,  an  increase  in  the 
desirability  of  Characteristic  A  may  lead  to  an  increase 
in  the  desirability  of  Characteristic  B  and  vice  versa); 
although  these  pairs  may  be  important,  they  do  not 
represent  true  tradeoffs. 

The  following  scenarios  were'  used  in  Step  Five  to 
explore  the  potential  for  tradeoffs  in  the  system  under 
study. 

1)  Ownship  State/Accuracy  of  Pilots  Current  World 
Model  Tradeoff  scenarios,  per  se,  are  difficult  to  imagine 
for  these  two  characteristics.  Specifically,  one  would  not 
necessarily  expect  these  two  characteristics  to  be  inversely 
related.  Instead,  a  direct  relation  would  be  expected.  For 
example,  an  undesirable  aircraft  attitude  might  affect  the 
pilot’s  ability  to  obtain  information  from  the  world  (and 
therefore  limit  the  accuracy  of  the  pilots  current  world 
model).  On  the  other  hand,  an  inaccurate  pilot  world 
model  might  erroneously  alter  the  control  actions  of  the 
pilot  (and  create  an  undesirable  state  of  ownship).  While 
this  relation  appears  to  be  a  direct  relation,  the  pairing 
of  these  two  characteristics  draws  attention  to  what  is 
probably  an  important  synergistic  relation. 

2)  Ownship  State! Accuracy  of Current  Machine  Model 
Like  the  previous  pairing,  an  inverse  relation  between 
these  two  characteristics  is  difficult  to  imagine.  Once 
again,  a  direct  relation  would  be  expected.  For  example, 
an  undesirable  aircraft  state  (e.g.,  it  is  too  close  to  ter¬ 
rain)  would  result  in  the  pilot  attending  to  information 
other  than  that  presented  by  the  CDTI.  Therefore,  an  • 
undesirable  aircraft  state  would  diminish  the  quality  of 
the  pilot’s  machine  model,  because  the  pilot  presumably 
would  have  less  knowledge  of  the  machine  state  than  would 
otherwise  be  the  case.  On  the  other  hand,  an  inaccurate 
machine  model  might  affect  ownship  state,  in  that  the 
pilot’s  actions  would  not  be  based  on  all  relevant  informa¬ 
tion.  Although  the  pair  does  not  represent  a  tradeoff,  the 
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pairing  of  these  two  characteristics  again  draws  attention 
to  what  is  probably  an  important  synergistic  relation, 

3)  Ownship  State/Perceived  Time  Pressure .  Tradeoff 
scenarios  are  easily  imagined  for  this  pairing.  For  example, 
when  ownship  is  at  an  undesirable  altitude,  speed,  or  at¬ 
titude,  the  pilot  would  feel  at  least  some  time  pressure  (Le,, 
as  the  acceptability  of  ownship  state  decreases,  perceived 
time  pressure  increases).  If  pressured  by  time,  the  pilot  may 
make  errors  that  worsen  ownship  state  (i.e.,  as  perceived 
time  pressure  increases,  the  acceptability  of  ownship  state 
may  decrease).  This  pairing  of  characteristics  represents 
a  true  tradeoff. 

4)  Ownship  State/Level  of  Pilot  Mental  Workload.  Men¬ 
tal  workload  is  different  than  the  characteristics  discussed 
thus  far;  high  mental  workload  is  an  ^desirable  state. 
Given  this  distinction,  this  pairing  does  not  appear  to 
represent  a  tradeoff.  For  example,  if  aircraft  altitude  were 
unacceptable  (i.e. ,  the  acceptability  of  ownship  state  were 
low),  then  pilot  mental  workload  would  increase  in  the 
attempt  to  diagnose  the  problem  or  simply  attain  the 
correct  altitude  (i.e,,  pilot  mental  workload  also  would 
be  in  an  undesirable  state).  On  the  other  hand,  as  pilot 
mental  workload  increases  (i.e.  reaches  an  undesirable 
state),  the  pilot  is  more  likely  to  ignore  some  responsibili¬ 
ties.  Therefore,  the  acceptability  of  ownship  state  could 
become  undesirable  as  mental  workload  increases.  In 
most  cases,  this  pair  probably  has  a  direct  relation  (does 
not  represent  a  tradeoff).  However,  the  relation  can  be 
unstable  due  to  other  factors.  For  example,  a  compla¬ 
cent  pilot  would  not  necessarily  have  increases  in  mental 
workload  when  ownship  state  is  undesirable. 

5)  Ownship  State/Type  of  Action  Chosen  by  the  Pilot . 
The  pairing  does  not  appear  to  represent  a  tradeoff.  In¬ 
stead,  the  pair  has  a  direct  relation.  That  is,  as  the  suit¬ 
ability  of  the  chosen  action  decreases,  the  acceptability 
of  ownship  state  presumably  would  decrease.  Further,  as 
the  acceptability  of  ownship  state  decreases,  the  action 
chosen  by  the  pilot  may  be  less  desirable  (e.g.,  in  extreme 
cases,  the  pilot  may  panic  and  use  poor  judgment).  Again, 
lack  of  a  tradeoff  does  not  imply  anything  regarding  the 
importance  of  this  pairing. 

6)  Level  ofAir  traffic  Manger  sMentalWorkload/Level of 
Pilot MentalWorkload,  This  pairing  is  interesting  because 
scenarios  can  be  imagined  for  both  an  inverse  and  a  direct 
relation  between  the  characteristics.  If  the  pilot  performs 
most  of  the  duties  associated  with  self-separation,  the 
pilot  s  mental  workload  would  be  high  and  the  ATM  s 
mental  workload  would  be  relatively  low.  If  a  special  situ¬ 
ation  arose  (e.g.,  the  ADS-B  system  were  malfunctioning), 
the  ATM  s  mental  workload  would  increase  (as  the  ATM 
would  have  relatively  more  responsibilities).  At  the  same 
time,  the  pilot  s  mental  workload  would  decrease  (as  the 
pilot  would  have  relatively  less  responsibility).  A  direct 


relation  also  can  be  conceived  easily.  The  obvious  case  arises 
when  traffic  is  heavy.  A  pilot  may  request  assistance  from 
an  ATM.  However,  because  the  ATM  has  high  mental 
workload,  the  ATM  may  be  unable  to  assist  the  pilot  im¬ 
mediately,  As  a  result,  the  pilot  s  mental  workload  would 
be  relatively  high.  Similarly,  a  pilot  under  conditions  of 
high  mental  workload  may  not  respond  immediately  to 
an  ATM,  As  a  result,  the  ATM  s  mental  workload  may 
be  increased  by  the  pilot  s  high  mental  workload.  In  sum, 
these  characteristics  may  sometimes  represent  a  tradeoff, 
and  at  other  times,  may  have  a  direct  relation. 

7)  Level  ofAir  Traffic  Managers MentalWorkloadiType 
of  Action  Chosen  by  Pilot  This  pairing  of  characteristics 
does  not  represent  a  tradeoff.  When  the  action  chosen  by 
the  pilot  is  less  than  desirable  (e.g.,  the  pilot  unknowingly 
changes  heading  and  creates  the  potential  for  an  alert  zone 
violation),  the  ATM  s  mental  workload  might  increase 
(i.e,,  reach  an  undesirable  state),  because  it  is  the  ATM  s 
duty  to  supervise  the  actions  chosen  by  pilots.  On  the 
other  hand,  if  an  ATM  has  high  mental  workload,  the 
ATM  might  not  be  as  effective  at  noticing  a  potential 
conflict.  Therefore,  the  pilot  may  choose  an  action  with 
less  information  than  might  have  otherwise  been  avail¬ 
able.  Therefore,  when  the  ATM  s  mental  workload  is 
high  (i.e,,  in  an  undesirable  state),  there  is  a  better  chance 
that  the  action  chosen  by  the  pilot  also  will  be  less  than 
optimal. 

8)  Level  of  Confidence  in  Planned  Action/Level  of  Pilot 
Workload .  This  pairing  does  not  represent  a  tradeoff.  As 
the  pilot  s  level  of  confidence  in  the  planned  action  goes 
down  (Le.,  reaches  a  relatively  less  desirable  state),  the 
pilot  will  search  the  environment  in  an  attempt  to  con¬ 
firm  or  disconfirm  the  adequacy  of  the  planned  action. 
Therefore,  the  pilot  s  mental  workload  will  increase  (will 
reach  a  relatively  less  desirable  state) .  On  the  contrary,  high 
pilot  workload  (an  undesirable  state)  would  decrease  the 
pilot  s  confidence  in  the  planned  action  (an  undesirable 
state).  Under  high  workload  conditions,  time  may  not 
allow  the  pilot  to  assess  all  relevant  information  and  leave 
the  pilot  feeling  less  confident. 

9)  Level of  Confidence  in  Planned Action/Type  of. 'Action 
Chosen  by  Pilot,  This  pair  of  characteristics  has  a  direct 
relation.  Typically,  a  high  level  of  confidence  yields  better 
performance.  Further,  when  the  most  appropriate  action 
is  chosen,  a  high  level  of  confidence  results.  The  only 
exception  to  this  rule  is  in  the  case  of  overconfidence. 
Specifically,  an  inverse  relation  occurs  when  the  confidence 
level  is  so  high  that  the  pilot  makes  uninformed  decisions 
(e.g.,  ignoring  crew  members),  and  these  decisions  can 
result  in  a  less  than  desirable  action. 

10)  Perceived  Time  Pressure/Type  of  Action  Chosen  by 
Pi^TTradeoffscenarios  are  easily  imagined  for  this  pairing. 
If  pressured  by  time,  the  pilot  may  make  decisions  too 
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quickly.  As  a  result,  the  chosen  action  may  be  less  than 
satisfactory.  Furthermore,  if  the  action  is  inadequate  (e.g., 
a  change  in  heading  that  results  in  an  alert  zone  conflict 
warning),  the  resulting  situation  may  need  to  be  rectified 
immediately. 

11)  Level  of  Pilot  Mental  Workload/Type  of  Action 
Chosen  by  Pilot.  This  pairing  does  not  appear  to  repre¬ 
sent  a  tradeoff.  As  pilot  mental  workload  increases  (i.e., 
reaches  a  less  desirable  state),  the  pilot  is  more  likely  to 
overlook  relevant  information.  Therefore,  as  workload 
becomes  less  desirable,  the  quality  of  the  chosen  action 
may  decrease.  Furthermore,  if  the  type  of  action  chosen 
is  unacceptable  (e.g.,  aircraft  is  headed  into  terrain),  then 
pilot  mental  workload  naturally  would  increase  (become 
less  desirable)  in  an  attempt  to  correct  the  mistake. 

In  sum,  the  goal  of  Step  5  was  to  identify  tradeoffs 
that  occur  among  system  characteristics.  In  all,  62  cases 
were  identified  in  which  there  is  the  potential  for  a  trad¬ 
eoff.  Due  to  the  large  number  of  cases,  the  analysis  was 
limited  to  include  only  the  tradeoffs  associated  with  the 
13  most  important  characteristics  that  were  identified  in 
Step  Four.  These  13  characteristics  yielded  11  cases  of 
potential  tradeoffs.  Plausible  scenarios  were  generated 
to  determine  which  of  these  potential  tradeoffs  are  truly 
tradeoffs.  Two  of  the  11  pairs  appear  to  represent  true 
tradeoffs  in  the  system  (i.e.,  they  are  inversely  related), 
while  six  pairs  appear  to  be  directly  related.  Three  pairs 
appear  to  have  unstable  relations,  suggesting  that  the 
circumstances  are  important  in  determining  whether 
they  represent  a  tradeoff  or  a  direct  relation.  Although 
there  were  only  two  tradeoffs,  an  important  relation  was 
shown  for  every  pair  of  characteristics. 

4.6  Step  6:  Identify  Information  Requirements 

The  final  step  in  the  current  analysis  is  an  identifica¬ 
tion  of  information  requirements.  First,  the  original 
matrix  is  used  to  identify  errors  in  the  system  that  lead 
to  failures,  incidents,  or  accidents  (Riley,  personal  com¬ 
munication,  June  1,  1999).  For  each  potential  error,  an 
imagined  scenario  is  documented.  Once  these  scenarios 
are  written,  the  analyst  asks,  'What  information  would 
assist  the  operator  in  preventing,  detecting,  or  correcting 
the  possible  error?”  Responses  to  this  question  result  in 
a  list  of  information  requirements. 

The  creation  of  a  scenario  for  each  of  the  767  pairings 
in  the  current  matrix  is  beyond  the  scope  of  this  document 
(i.e.,  there  are  7 67  cell  entries  in  Appendix  C).  Therefore, 
only  the  important  characteristics,  identified  in  Step  Four, 
are  utilized  in  this  final  step.  Table  D5  of  Appendix  D 
presents  a  matrix  representing  only  the  interactions 
between  the  important  characteristics.  Note  that  this 
matrix  does  not  contain  a  negative  diagonal  because  the 


characteristics  on  the  left  and  top  of  the  matrix  are  not 
identical.  This  summary  matrix,  which  is  derived  from 
the  matrix  in  Appendix  C,  yielded  51  pairs  of  character¬ 
istics  that  interact.  Table  D6  of  Appendix  D  contains  the 
51  possible  scenarios  that  were  written  for  the  pairings 
identified  in  Table  D5.  Of  course,  these  scenarios  reflect 
only  possible  interactions  that  could  occur  in  a  free-flight 
traffic  environment  (i.e.,  the  environment  of  interest). 

Although  Riley  (1993)  suggests  asking  only  a  single 
question  to  determine  information  requirements,  the 
current  analysis  added  more  structure  to  the  task  by 
identifying  the  possible  error(s)  that  might  arise  in  each 
scenario  and  utilizing  three  questions  for  each  scenario. 
The  first  question  was  “  Whose  knowledge  of  the  situation 
would  assist  in  the  prevention,  detection,  or  mitigation 
of  the  error?”  In  this  step,  the  human  component  of  the 
system  was  emphasized.  The  second  question  was  “  What 
knowledge  would  assist  in  the  prevention,  detection,  or 
mitigation  of  the  error?”  The  scenarios  are  more  helpful  in 
identifying  the  required  (i.e.,  necessary)  knowledge  rather 
than  the  required  information.  However,  identifying  the 
required  knowledge  is  important.  Available  information 
is  a  necessary,  but  not  sufficient,  condition  for  knowledge, 
and  the  ultimate  goal  is  to  assist  the  pilot  in  obtaining 
knowledge.  Once  the  necessaryknowledgewasidentified, 
the  third  question  was  posed:  “  What  information  is  neces¬ 
sary  for  the  appropriate  person  to  obtain  the  necessary 
knowledge?”  The  resulting  information  requirements  for 
the  current  system  are  categorized  according  to  the  three 
aforementioned  questions,  and  they  appear  in  Table  D7 
of  Appendix  D. 

The  current  analysis  yielded  a  total  of  1 00  information 
requirements.  Because  the  information  requirements  for 
the  human  in  the  system  (i.e.,  the  pilot  of  ownship)  and 
the  requirements  for  the  pilots  in  the  world  were  identi¬ 
cal,  these  information  requirements  were  combined  into 
one  category  (i.e.,  “Pilot”).  Of  the  100,  67  information 
requirements  were  for  the  air  carrier  pilot  in  the  free- 
flight  environment,  and  33  information  requirements 
were  identified  for  the  air  traffic  manager  in  the  free-flight 
environment.  However,  one  must  be  cognizant  that,  in 
this  application  of  the  FAIT  analysis,  the  ATM  is  merely 
part  of  the  world  in  this  system  (i.e.,  the  ATM  is  not  part 
of  the  human-machine  system  that  is  of  primary  inter¬ 
est).  There  probably  would  be  many  more  information 
requirements  identified  for  the  ATM  if,  at  the  initial  step  ‘ 
of  the  FAIT  analysis,  the  manager  was  identified  as  the 
human  of  primary  interest. 

The  results  obtained  via  the  FAIT  analysis  were 
compared  with  those  obtained  from  traditional  task 
analyses.  As  discussed  in  the  introduction,  Endsley  et  al. 
(1998)  performed  a  task  analysis  to  identify  information 
requirements  for  the  commercial  airline  pilot.  Endsley 


and  Rodgers  (1994)  also  performed  a  similar  task  analysis 
for  an  en  route  air  traffic  controller.  As  discussed  earlier, 
one  of  the  present  goals  was  to  compare  the  results  of 
these  two  studies  with  the  current  findings. 

While  the  precise  names  given  to  the  information 
requirements  varied  slightly,  28  of  the  100  information 
requirements  identified  by  the  FAIT  are  identical  to  those 
found  by  either  Endsley  et  al.  (1998)  or  Endsley  and 
Rodgers  (1994).  These  28  information  requirements  are 
shaded  gray  in  Table  D7.  The  combined  total  of  informa¬ 
tion  requirements  in  theEndsely  etal.  (1998)  and  Endsley 
and  Rodgers  (1994)  references  exceeds  100.  This  amount 
is  not  surprising,  given  that  they  did  not  limit  themselves 
to  the  task  of  traffic  avoidance.  For  example,  the  list  of  in¬ 
formation  requirements  in  the  present  analysis  recognizes 
the  importance  of  terrain.  However,  because  the  CDTI 
is  the  only  machine  in  the  system  under  consideration, 
the  only  pertinent  information  is  an  alert  that  refers  the 
pilot  to  another  system.  If  the  current  analysis  included 
all  information  required  for  piloting  (as  did  Endsley  et 
al.),  the  overall  list  of  information  requirements  would 
be  more  extensive. 

Given  that  only  28  of  the  information  requirements 
identified  in  the  FAIT  analysis  were  redundant  with  that 
of  Endsley  etal.(1998)  and  Endsley  and  Rodgers  { 1 994) , 
the  remaining  72  information  requirements  are  unique 
to  the  analysis.  The  identification  of  unique  information 
requirements  is  the  result  of  at  least  four  factors.  First, 
unlike  the  research  performed  by  Endsley,  the  current 
analysis  was  not  “technology-free.”  In  fact,  the  FAIT 
analysis  is  necessarily  technology-inclusive.  Consequendy, 
some  information  requirements  are  very  specific  to  the 
CDTI.  For  example,  Endsley  et  al.  (1998)  list  two  gen¬ 
eral  information  requirements  for  the  commercial  airline 
pilot:  “time  available  to  perform  tasks”  and  “projected 
time  until  maneuver  required.”  An  analogous,  yet  qualita¬ 
tively  different,  information  requirement  identified  in  the 
present  analysis  is  “time  until  the  next  alert  zone  contact 
watch/warning.”  The  latter  piece  of  information  would 
allow  the  pilot  to  assess  how  much  time  can  be  spent 
on  tasks  other  than  traffic  avoidance  and  the  amount  of 
time  before  a  traffic  avoidance  might  become  an  issue 
(i.e.,  the  time  until  there  is  a  watch  or  warning).  In  other 
words,  displaying  “time  until  the  next  alert  zone  contact 
watch/warning”  would  allow  the  pilot  to  determine  the 
amount  of  time  the  CDTI  can  be  ignored. 

The  second ,  related  reason  the  current  analysis  yields 
unique  results  is  that  it  assumes  some  level  of  automa¬ 
tion.  For  example,  Endsely  et  al.  (1998)  listed  “projected 
separation  between  aircraft”  as  an  information  require¬ 
ment,  whereas  the  FAIT  analysis  identified  “alert  zone 
contact  watch/warning”  as  an  information  requirement. 
In  other  words,  Endsley  et  al.  examined  the  current  flight 


environment,  in  which  the  pilot  is  responsible  for  taking 
raw  data  (i.e.,  separation  between  aircraft)  and  formulat¬ 
ing  predictions  regarding  future  separation.  In  contrast, 
the  current  analysis  places  the  responsibility  of  prediction 
upon  the  CDTI.  Therefore,  in  a  free-flight  environment 
with  a  CDTI,  the  pilot  merely  needs  to  know  if  them  is 
a  watch  (i.e. ,  a  moderate  possibility  of  a  future  alert  zone 
conflict)  or  a  warning  (i.e.,  high  probability  of  a  future 
alert  zone  contact).  Rather  than  making  predictions,  the 
pilot  needs  to  respond  appropriately  when  the  CDTI 
makes  predictions  of  impending  contact. 

The  third,  and  related,  reason  the  current  analysis  yields 
unique  results  is  that  it  examines  a  free-flight  environment. 
Unlike  the  previous  studies,  the  current  analysis  places 
more  traffic  avoidance  responsibility  on  the  pilot  (and 
the  CDTI)  than  on  an  air  traffic  manager.  For  example, 
Endsley  and  Rodgers  (1994)  dedicate  several  information 
requirements  to  the  ATM  acquiring  knowledge  regarding 
the  impact  ofproposed  changes  (e.g..  What  would  happen 
if  Aircraft  X  increases  altitude  by  1,000  feet?).  However, 
the  current  research  categorizes  somewhat  analogous 
information  as  being  requirements  for  the  pilot  (i.e., 
“hypothetical  results  of  planned  changes”). 

The  fourth  reason  the  current  analysis  yields  unique 
results  is  that  unlike  most  traditional  task  analyses,  the 
FAIT  analysis  encourages  the  analyst  to  simultaneously 
consider  specific  components  of  the  systems  (e.g.,  char¬ 
acteristics  of  the  CDTI)  and  the  manner  in  which  these 
components  interact  (e.g.,  characteristics  of  the  CDTI 
that  might  affect  the  pilot,  characteristics  of  the  pilot 
that  might  affect  the  CDTI ,  characteristics  of  the  world 
that  might  affect  the  pilot,  etc.).  In  addition,  identify¬ 
ing  the  information  flow  in  the  system  encourages  the 
consideration  of  the  real-time  operation  of  the  system. 
The  creation  of  possible  scenarios  further  enhances  such 
consideration,  in  that  the  scenarios  require  deliberation 
regarding  the  effects  of  particular  circumstances  on  the 
system  when  it  is  operating. 

4,7  Concluding  Remarks  Regarding  the  Analysis 

The  current  research  used  the  FAIT  analysis  to  (1) 
estimate  the  relative  importance  of  characteristics  in  a 
system,  (2)  identify  tradeoffs  in  a  system,  and  (3)  identify 
information  requirements.  However,  the  FAIT  Users 
Manual  (Riley,  1993)  actually  presents  five  different 
functional  applications.  The  two  options  not  applied 
here  are  related  to  the  development  of  issues  documents 
and  requirements  documents. 

Several  factors  influenced  the  decision  to  exclude 
the  issues  and  requirements  documents  in  favor  of  the 
other  options.  The  primary  reason  is  that  the  issues  and 
requirements  are  determined  from  die  interactions  in  the 
matrix.  Specifically,  creation  of  an  issues  or  a  requirement 
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document  requires  the  creation  of  a  scenario  for  every 
entry  found  in  the  matrix.  The  number  of  interactions 
identified  in  the  current  research  (767  pairings)  made 
these  options  impractical.  Another  option,  in  the  cur¬ 
rent  analysis,  would  be  to  examine  only  the  issues  and 
requirements  associated  with  the  important  characteristics 
identified  in  Step  4.  Many  scenarios  were  written  during 
the  process  of  identifying  tradeoffs  in  the  system  (Step 
5)  and  identifying  information  requirements  (Step  6). 
These  scenarios  are  adequate  in  identifying  the  important 
issues  and  requirements. 


The  current  research  utilizes  another  option  that  Riley 
(1993)  implies.  Specifically,  Table  3  is  a  reproduction 
of  a  figure  presented  in  the  FAIT  Users  Manual  that 
allows  for  an  identification  of  the  types  of  issues  that 
arise  in  the  current  system  and  the  relative  occurrence 
of  each  type. 

Given  the  classification  in  Table  3,  totals  were  calcu¬ 
lated  for  each  part  of  the  matrix  and  are  presented  in 
Table  4.  Table  4  suggests  that  the  operator  (the  pilot) 
has  the  most  influential  characteristics  in  the  system. 
Although  not  quite  as  great,  the  characteristics  of  the 


Table  3.  Classification  of  issues  based  on  the  layout  of  the  matrix  (adapted  from 
Riley,  1993). 


Receivers 


Environment 

Operator 

Machine 

Environment 

Training  Issues 

Operator 

Training  Issues 

Operator-Driven 
System  Design 
Issues 

Machine 

Training  Issues 

Automation  Issues 

Table  4.  Number  of  interactions  (%  of  interactions)  found  in  each  section  of  the  matrix. 

Receivers 


Environment 

Operator 

Machine 

Total 

Influence 

Environment 

70 

180 

46 

296 

•  (9%) 

(23%) 

(6%) 

(38%) 

Operator 

56 

(7%) 

262 

(34%) 

29 

(4%) 

347 

(45%) 

Machine 

33 

71 

20 

124 

(4%) 

(9%) 

(3%) 

(16%) 

Total 

159 

513 

95 

Grand  Total: 

Sensitivity 

(21%) 

(67%) 

(12%) 

767 

'  (100%) 
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environment  have  substantial  influence  in  the  system, 
and  the  machines  (CDTIs)  characteristics  has  the  least 
amount  of  influence  in  the  system.  The  operator,  *by 
far,  has  the  most  sensitive  characteristics  in  the  system. 
The  characteristics  of  the  environment  are  substantially 
less  sensitive  than  the  operator  s  characteristics,  and  the 
machines  characteristics  are  the  least  sensitive  of  the 
three.  Therefore,  the  characteristics  of  the  pilot  have 
the  most  influence  on  the  system,  and  at  the  same  time, 
the  pilots  characteristics  are  more  vulnerable  than  any 
other  component  of  the  system. 

The  information  from  Table  3  and  Table  4  are  com¬ 
bined  in  Table  5,  which  identifies  the  types  of  issues  that 
arise  in  the  current  system  and  the  relative  occurrence 
of  each  type.  Few  operator-driven  system  designs  were 
identified.  However,  this  finding  may  not  be  surprising, 
given  that  the  NASA-AMES  CDTI  has  only  moderate 
'autonomy 5  and  “intelligence55  (i.e.,  it  is  a  “personalized, 
simple  aid”).  Therefore,  one  would  not  expect  operator 
characteristics  to  have  a  great  amount  of  influence  on 
the  characteristics  of  the  CDTI,  nor  would  one  expect 
the  characteristics  of  the  machine  to  be  very  vulnerable 
to  the  characteristics  of  the  operator.  Few  automation 
issues  were  identified,  and  again  this  finding  is  probably 
because  the  NASA-AMES  CDTI  is  only  moderately 
“autonomous55  and  “intelligent.55  The  results  shown  in 
Table  5  suggest  that  training  issues  are  quite  important 
in  the  system.  As  already  stated,  it  is  clear  that  operator 
characteristics  are  the  most  vulnerable  characteristics  in 
the  system.  While  operator  characteristics  interact  with 
characteristics  from  all  three  components  of  the  system 
(i.e.,  the  environment,  the  operator,  and  the  machine), 
the  majority  of  interactions  are  environment/operator 
and  operator/operator  interactions.  Referring  back  to 
Table  4,  note  that  over  half  of  the  entries  fall  in  the  cells 
where  “Environment55  is  the  driver  and  “Operator55  is  the 
receiver  (23%),  and  where  “Operator55  is  the  driver  and 
“Operator55  is  the  receiver  (34%).  This  finding  suggests 


that  training  would  be  most  cost-effective  if  the  major¬ 
ity  of  training  for  free  flight  were  spent  concentrating 
on  characteristics  of  the  environment  and  the  operator. 
While  free  flight  introduces  a  new  piece  of  technology  (i.e. , 
the  CDTI),  this  analysis  suggests  that  it  is  not  necessar¬ 
ily  the  equipment  that  is  cause  for  concern.  Instead,  the 
human  factors  issues  (especially  as  they  relate  to  traffic 
awareness)  lie  in  the  novel  procedures,  types  of  human 
interactions,  and  environment.  This  suggestion  is  further 
supported  by  the  findings  presented  earlier.  Almost  every 
important  characteristic  that  was  identified  in  Step  4  is  a 
characteristic  of  the  environment  or  the  operator;  these 
findings  will  now  be  reviewed  in  terms  of  their  practical 
implications. 

5.0  DISCUSSION  AND  IMPLICATIONS 
OF  THE  FINDINGS 

5. 1  Summary  of  Findings 

The  system  of  interest  is  one  that  includes  a  pilot,  a 
free-flight  traffic  environment,  and  a  CDTI.  The  FAIT 
analysis  yielded  seven  characteristics  that  are  highly  influ¬ 
ential  m  the  system:  weather, general piloting  skills,  time  of 
day,  terrain ,  ownship  state  (altitude,  attitude,  speed,  etc.), 
level  of pilot  mental  workload,  and  perceived  time  pressure. 
Because  these  characteristics  are  highly  influential,  they 
have  a  great  impact  on  the  functioning  of  the  system  as 
a  whole.  Specifically,  when  any  of  these  seven  character¬ 
istics  are  in  an  undesirable  state,  they  have  the  ability  to 
negatively  affect  many  other  characteristics  in  the  system. 
Therefore,  in  the  process  of  designing  technology  (e.g.,  a 
CDTI)  and  training  programs  for  a  free-flight  environ¬ 
ment,  these  seven  characteristics  should  be  emphasized. 
Specifically,  efforts  should  be  aimed  at  increasing  the 
chances  that  these  characteristics  will  achieve  and  remain 
at  a  desirable  state.  Of  course,  some  of  these  character¬ 
istics  (i.e.,  weather,  time  of  day,  and  terrain)  cannot  be 
controlled.  In  such  cases,  the  design  of  technology  and 


Table  5.  Types  of  issues  that  arise  in  the  current  system  and  the  relative  occurrence  of  each 
type. 


Training  Issues 

Operator- 
Driven  System 
Design  Issues 

Automation 

Issues 

Miscellaneous 

Issues 

Number  of 
Cases 

513 

29 

20 

205 

Percent  of 
Total  Cases 
(Cases/767) 

67% 

i 

4% 

3% 

27% 

15 


training  programs  should  assist  in  ameliorating  the  un¬ 
desirable  effects  of  these  characteristics. 

Seven  characteristics  were  identified  as  being  highly 
sensitive  in  the  free-flight  traffic  environment:  type  of  action 
chosen  by  the  pilot ,  level  of  pilot  mental  workload \  appropri¬ 
ateness  of  planned  action ,  ownship  state ,  level  of  air  traffic 
managers  ’  mental  workload \  accuracy  of  current  machine 
model ,  and  level  of  confidence  in  planned  action.  These 
characteristics  are  more  vulnerable  than  other  character¬ 
istics  of  the  system.  Therefore,  in  the  process  of  designing 
technology  (e.g.,  a  CDTI)  and  training  programs  for  a 
free-flight  environment,  these  seven  characteristics  also 
should  be  emphasized  to  decrease  their  vulnerability  when 
related  characteristics  are  in  undesirable  states. 

A  separate  analysis  on  traffic  awareness  showed  that  it 
was  not  a  very  influential  characteristic.  However,  it  was 
one  ofthe  most  sensitive  characteristics  in  the  system,  and 
therefore,  technology  and  training  should  be  designed  to 
aid  in  decreasing  the  vulnerability  of  traffic  awareness. 

In  general,  special  attention  should  be  given  to  cases 
in  which  the  highly  influential  characteristics  interact 
with  the  highly  sensitive  characteristics.  The  present 
analysis  suggested  that,  in  a  free-flight  environment  with 
a  CDTI,  there  are  51  cases  where  highly  influential  and 
highly  sensitive  characteristics  interact.  Furthermore,  two 
characteristics  of  the  system  proved  to  be  both  highly 
influential  and  highly  sensitive  (ownship  state  and  level 
of  pilot  mental  workload).  Therefore,  ensuring  that  these 
two  characteristics  remain  in  a  desirable  state  would  be 
of  utmost  importance  when  considering  the  surveillance 
activities  associated  with  self-separation. 

Eleven  potential  tradeoffs  were  identified  for  the 
system.  Only  two  of  these  cases  resulted  in  a  pair  of 
characteristics  being  inversely  related.  However,  pairs  of 
characteristics  that  are  directly  related  also  are  important 
because  each  of  these  pairs  represents  a  case  in  which  the 
first  characteristic  influences  the  second  and  vice  versa 
(be  it  inversely  or  directly).  Therefore,  the  interaction  of 
all  1 1  pairs  of  characteristics  should  be  considered  in  the 
design  of  tools  or  procedures  to  be  used  in  the  free-flight 
environment. 

One  ofthe  goals  of  the  present  research  was  to  identify 
and  classify  information  requirements  for  pilot  surveil¬ 
lance  functions  in  the  air  carrier.  One  hundred  information 
requirements  were  identified,  and  67  of  these  information 
requirements  were  for  pilots  in  the  system.  The  remain¬ 
ing  33  information  requirements  were  for  ATMs.  These 
information  requirements  might  be  helpful  in  the  pro¬ 
cess  of  designing  new  technologies  for  pilots  and  ATMs. 
Specifically,  these  100  information  requirements  may  be 
utilized  in  the  early  phases  of  the  design  process.  The  list 
could  be  used  to  ensure  that  the  necessary  information 


is  available  to  the  appropriate  party.  Without  such  in¬ 
formation,  the  pilot  (or  ATM)  will  be  unable  to  obtain 
the  knowledge  necessary  for  traffic  awareness. 

Finally,  when  compared  with  “operator-driven  system 
design  issues”  and  “automation  issues,”  the  current  analysis 
indicates  that  training  may  be  the  most  important  issue 
to  address  in  the  free-flight  traffic  environment. 

5.2  The  Utility  ofthe  FAIT  Analysis 

The  second  goal  of  the  current  analysis  was  to  exam¬ 
ine  the  utility  ofthe  FAIT  analysis.  Like  traditional  task 
analyses,  the  FAIT  analysis  allows  researchers  to  identify 
information  requirements.  However,  the  FAIT  analysis 
has  value  beyond  that  of  traditional  task  analyses  because 
it  yields  additional  information.  Specifically,  the  FAIT 
analysis  allows  the  analyst  to  examine  the  relative  influence 
and  the  relative  sensitivity  of  important  characteristics 
within  the  system,  as  well  as  identify  potential  system 
tradeoffs.  The  FAIT  analysis  also  encourages  the  researcher 
to  emphasize  both  the  various  system  components  (i.e., 
the  environment,  the  human,  and  the  machine)  and  the 
system  as  a  whole.  Third,  the  FAIT  analysis  also  recognizes 
the  importance  of  considering  real- ti  me  system  operation 
(via  scenarios),  and  it  allows  the  researcher  to  include 
even  the  psychological  aspects  of  the  human  in  the  sys¬ 
tem  (e.g.,  mental  workload).  Finally,  the  FAIT  analysis 
is  a  useful  tool  for  examining  the  effects  of  a  particular 
technology  in  that  it  yields  results  that  are  specific  to  a 
task  environment. 

The  current  research  included  several  novel  procedures 
that  proved  quite  useful  in  adding  structure  to  the  FAIT 
analysis.  First>  a  formal  definition  of  the  term  “charac¬ 
teristic”  was  composed,  and  second ,  a  formal  and  unique 
question  was  used  to  identify  characteristics  for  each  node 
(cf.,  Appendix  B).  Third ,  a  more  structured  analysis  was 
used  to  extract  information  requirements  from  scenarios. 
Specifically,  rather  than  asking  one  question  about  the 
scenario,  a  series  of  steps  was  performed.  Fourth ,  a  method 
by  which  the  analyst  can  limit  the  analysis  was  introduced. 
Specifically,  the  90th  percentile  was  used  in  several  steps 
to  identify  the  important  characteristics  from  a  large 
pool  of  characteristics.  Riley  (personal  communication, 
May,  1999)  arbitrarily  recommends  limiting  the  overall 
number  of  characteristics  to  50.  However,  because  the 
FAIT  analysis  is  meant  for  complex  systems,  it  is  quite 
likely  that  the  number  of  characteristics  often  will  exceed  ’ 
50.  In  future  research,  analysts  may  opt  to  utilize  the  90th 
percentile  as  a  cutoff  when  the  system  is  too  complex  to 
limit  the  characteristics  to  50.  A  fifth  and  related  point 
pertains  to  the  identification  of  information  requirements. 
The  current  research  utilized  only  the  top  10%  of  char¬ 
acteristics  in  the  analysis,  and  while  the  scenarios  were 
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surprisingly  helpful,  most  information  requirements  were 
identified  after  addressing  the  top  20  scenarios.  Therefore, 
in  the  analysis  of  complex  systems,  utilizing  only  the  top 
5%  of  characteristics  in  the  identification  of  information 
requirements  may  be  adequate. 

6,0  FUTURE  RESEARCH 

Future  research  is  needed  in  three  areas.  First,  the 
current  analysis  requires  validation  from  domain  experts. 
Input  from  domain  experts  would  be  invaluable.  Their 
input  could  be  obtained  readily  using  the  steps  of  the  cur¬ 
rent  analysis,  that  have  been  documented  here.  Domain 
experts  would  be  invaluable  in  the  validation  process,  as 
they  could  offer  information  that  might  further  enhance 
the  current  application  of  the  FAIT  analysis.  Specifically, 
domain  experts  might  be  queried  regarding  the  impor¬ 
tance  of  characteristics.  They  might  also  be  asked  about 
the  frequency  with  which  characteristics  influence  one 
another.  Such  information  could  be  easily  obtained 
through  some  sort  of  ranking  procedure,  or  by  using 
a  Likert  scale.  Thereafter,  the  row  and  column  totals 
found  in  the  matrix  could  be  weighted  according  to  their 
importance  and  frequency  of  influence.  In  its  current 
form,  the  FAIT  analysis  does  not  take  the  Importance  or 
frequency  of  occurrence  Into  account.  Second ,  the  FAIT 
analysis  should  be  applied  to  other  tasks  associated  with 
surveillance  functions  (e.g.,  weather  avoidance).  Such  an 
analysis  would  ensure  that  all  information  requirements 
have  been  identified  and  would  provide  further  insight 
into  the  most  important  characteristics  of  the  free-flight 
system  as  a  whole.  Finally ,  the  FAIT  analysis  should  be 
extended  to  include  other  piloting  tasks.  With  applications 
of  the  FAIT  analysis  to  multiple  tasks,  a  fair  comparison 
can  be  made  between  the  FAIT  analysis  and  traditional 
task  analyses  (e.g.,  Endsley  et  al,  1998). 
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8.0  APPENDIX  A 

FUNCTIONAL  DESCRIPTION  OF  NASA-AMES  CDTI 


Figure  A1  presents  a  snapshot  of  the  CDTI  simula¬ 
tion  that  was  used  for  the  current  analysis.  This  figure 
represents  the  candidate  CDTI  in  its  most  basic  form. 
Regardless  of  operator  inputs,  there  are  several  aspects  of 
the  display  that  remain  constant  in  this  simulation; 

1)  The  pilot  s  ownship  is  at  the  bottom,  center  of  the 
screen.  (Here,  it  is  represented  by  a  filled,  white 
chevron.) 

2)  The  upper  portion  of  the  display  presents  a  compass 
rose. 

3)  The  upper,  center  of  the  display  presents  the  pilot  s 
current  magnetic  heading  (e.g.,  “336”). 


4)  A  dotted  white  line  extends  from  ownship  and 
aligns  with  the  current  magnetic  heading  on  the 
compass. 

5)  A  shaded  solid  line  (which  is  colored  magenta  in 
the  actual  display)  also  extends  from  ownship  and 
represents  the  current  route  of  ownship. 

6)  Shaded,  star-like  shapes  (colored  magenta  in  the  ac¬ 
tual  display)  represent  way  points  and  have  shaded 
labels  attached  to  them  (e.g.,  “OAL  -099”). 

(a)  The  filled,  star-like  shape  represents  the  active 
waypoint. 

(b)  Unfilled,  star-like  shapes  represent  waypoints 
to  be  reached  in  the  future. 


Figure  A1 .  The  version  of  NASA’s  CDTI  that  was  used  in  the  current  analysis. 

A1 


7)  The  upper  right-hand  corner  of  the  display  presents 
the: 

(a)  active  waypoint  (e.g.,  “OAL  -099”), 

(b)  time  to  the  active  waypoint  (Note:  here  it  reads 
“OOOO.Oz,”  because  the  simulation  requires  adjunct 
software  to  obtain  this  information.  Normally,  the 
time  to  reach  OAL  -099  would  be  inserted  in  this 
field),  and 

(c)  nautical  miles  to  the  active  waypoint  (Again, 
here  it  reads  0.0  NM,”  because  additional  software 
is  needed  to  obtain  this  value). 

8)  The  upper  left-hand  corner  of  the  screen  displays 
the  pilots: 

(a)  current  ground  speed  (e.g.,  “GS  491”), 

(b)  true  air  speed  (e.g.,  “TAS  491”),  and 

(c)  altitude  (or  flight  level)  in  hundreds  of  ft.  (e.g., 
“FL400”  represents  an  altitude  of  40,000  ft.). 

9)  The  display  represents  1 60  miles  of  airspace,  and  80 
miles  of  airspace  is  denoted  by  the  center  tick  mark 
labeled  “80.” 

10)  The  very  bottom  of  the  display  contains  a  toolbar 
thdt  allows  the  pilot  to  manipulate  the  CDTI. 

Outside  of  any  manipulations  the  pilot  can  perform, 
there  are  certain  aspects  of  the  display  that  change 
with  circumstances: 

1)  Under  normal  circumstances,  aircraft  are  coded  ac¬ 
cording  to  their  altitude. 

(a)  Ownship  is  coded  white. 

(b)  Aircraft  at  ownship  s  altitude  are  coded  white. 

(c)  Aircraft  below  ownship  are  coded  brown. 

(d)  Aircraft  above  ownship  are  coded  blue. 

2)  Aircraft  within  a  +/-  6,000  ft  range  of  ownship  au¬ 
tomatically  have 

(a)  their  altitudes  displayed  as  a  tail  tag. 

(b)  a  one-minute  predictor  line  extending  from 
the  aircraft  symbol.  (This  is  the  case  for  ownship  as 
well) 

3)  An  arrow  (^or  Np)  appears: 

(a)  on  the  upper  left-hand  corner  of  the  display 
(i.e.,  aside  the  altitude  reading)  when  ownship  is 
ascending  or  descending. 

(b)  next  to  any  other  aircraft  that  is  ascending  or 
descending 

4)  When  a  potential  for  alert  zone  contact  exists,  the 
display  changes  in  several  ways.  These  changes  are 
listed  below  and  can  also  be  seen  in  Figure  A2. 

(a)  an  alert  message  appears  on  the  bottom,  left- 
hand  corner  of  the  screen.  This  message  presents  the 


estimated  time  to  closest  approach  (e.g.,  “ALERT 
CTA:  2:17”). 

(b)  the  datatag  associated  with  the  intruder  aircraft 
is  automatically  displayed. 

(c)  both  the  ownship  and  the  intruder  aircraft  are 
coded  differently  than  normal. 

(1)  The  chevron  representing  the  aircraft  is 
coded  (in  yellow  on  the  actual  display). 

(2)  On  the  actual  display,  a  (yellow)  line  extends 
from  the  aircraft  symbol  with  a  2.5  nm  circle  at  the 
end  of  it.  (Overlap  of  two  2.5  nm  circles  results  in  the 
aircraft  s  5  nm  alert  zone  being  contacted.)  However, 
the  portion  of  the  one-minute  predictor  line  is  coded 
white. 

NASA’s  CDTI  provides  the  pilot  with  many  options. 
In  this  version  of  the  simulation,  all  changes  to  the  display 
are  performed  with  a  mouse.  All  options  available  on  the 
current  simulation  are  listed  below.  However,  they  are  in 
an  order  that  is  not  consistent  with  the  toolbar.  Instead, 
they  are  in  an  order  that  is  conducive  to  discussion. 

1)  Altitude  may  be  displayed  according  to  preference. 
The  pilot’s  current  selection  (e.g.,  Abs)  is  indicated 
on  the  toolbar  via  the  fourth  button  from  the  left. 
The  pilot  is  able  to  toggle  between  the  following  two 
options  by  simply  pressing  the  button. 

(a)  Absolute  altitude  (e.g.,  “420”  represents  42, 000 
ft.)  is  displayed  when  the  button  reads  “Abs.” 

(b)  Relative  altitude  (e.g.,  “+20”  represents  2,  000 
ft.  above  ownship)  is  displayed  when  the  button  reads 
“Rel.” 

2)  An  aircraft’s  complete  datatag  (containing  an  aircraft  s 
identification  number,  flight  level,  and  speed)  is  dis¬ 
played  when: 

(a)  an  aircraft  is  selected.  The  aircraft  symbol  turns 
green  upon  selection.  When  the  mouse  is  moved  away 
from  the  symbol,  the  aircraft  returns  to  its  previous 
color  and  the  datatag  disappears. 

(b)  the  IDs  button  is  selected.  In  this  case,  all  aircraft 
datatags  are  displayed  until  the  button  is  selected 
again. 

3)  Datatags  may  be  repositioned  by: 

(a)  selecting  a  particular  datatag  and  moving  it  with 
the  mouse. 

(b)  selecting  the  S  Tags  button.  In  this  case,  datatags 
will  be  moved  to  reduce  overlap  and  clutter.  (The 
S  Tags  button  will  not  affect  any  datatags  that  have 
been  moved  manually  by  the  pilot.) 

4)  Predictor  lines  can  be  displayed  for  all  aircraft  by 
selecting  the  Pred  button.  * 
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Figure  A2.  An  example  of  the  symbols  used  to  display  an  alert  zone  contact  on  NASA’s 
CDTI. 


5)  When  predictor  lines  are  displayed,  two  further  op-  mode,  the  aircraft  appears  to  “shoot”  these  pulses 

tions  are  available.  down  the  predictor  line.  This  option  is  helpful  if 

(a)  Selecting  the  third  button  from  the  right  changes  two  aircraft  have  predictor  lines  that  overlap.  When 

the  time  represented  by  the  predictor  lines.  The  time  in  this  mode,  selection  of  the  button  causes  the  lines 

represented  can  range  from  2  to  20  minutes  (in  2-  to  be  displayed  in  the  static  format. 

minute  intervals).  6)  The  Rtes  and  Cpt  buttons  do  not  have  functions  in 

(b)  The  format  of  the  predictor  lines  may  be  varied.  the  current  simulation. 

The  current  selection  is  indicated  on  the  toolbar  via  7)  The  RAT  button  offers  several  options,  but  these 
the  first  button  on  the  left.  options  are  more  complex  than  the  ones  previously 

(1)  The  predictor  lines  are  displayed  in  standard  described.  Therefore,  this  button  is  addressed  below 

format  when  the  button  reads  “Static.”  When  in  this  in  paragraph  form.  The  Undo  and  Enter  buttons  also 

mode,  selecting  this  button  causes  the  lines  to  be  are  discussed  in  the  context  of  the  RAT  button, 

displayed  in  pulsed  format. 

(2)  The  predictor  lines  are  displayed  in  a  pulsed 
format  when  the  button  reads  “Pulsed.”  When  in  this 
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8.1  The  Route  Assessment  Tool  (RAT) 

The  RAT  tool  essentially  serves  four  functions.  First, 
it  assists  the  pilot  in  assessing  the  effects  of  changes  in 
altitude  (i.e.,  it  allows  the  pilot  ask  “what  if?”).  Second, 
after  assessing  these  changes,  it  permits  the  pilot  to  accept 
or  reject  changes  in  altitude.  The  third  function  of  the 
RAT  tool  allows  the  pilot  to  assess  the  effects  of  changes 
in  course  (again,  it  allows  the  pilot  to  ask  "what  if?). 
Finally,  the  fourth  function  permits  the  pilot  to  accept 
or  reject  the  proposed  changes  in  course.  The  following 
paragraphs  describe  the  manner  in  which  these  four 
functions  are  used. 

Regardless  of  the  pilots  intentions  (i.e.,  changing  alti¬ 
tude  or  changing  course),  the  RAT  tool  is  always  activated 
by  simply  selecting  the  RAT  button.  After  selecting  the 
button,  a  minimized  window  appears  on  the  bottom, 
right-hand  corner  of  the  screen  (see  Figure  A3). 
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Figure  A3.  The  minimized  windowthatappears 
immediately  after  the  RAT  button  is  selected. 

The  arrow  on  the  right  of  this  window  (V)  is  always 
available  when  the  RAT  is  activated.  Whenever  this  ar¬ 
row  is  selected,  it  returns  the  window  to  this  minimized 
state  and  moves  the  window  to  an  even  lower  part  of  the 
screen.  Therefore,  the  purpose  of  this  arrow  is  to  declutter 
the  screen  when  necessary. 

To  use  the  RAT,  the  pilot  must  select  the  arrow  point- 
ing  right  (>).  When  this  arrow  is  selected,  the  menu 
appears  as  in  the  bottom,  right-hand  corner  of  Figure 
A 4.  Initially,  the  “null  point”  is  selected.  The  null  point 
represents  a  point  in  time/space  at  and  before  which  the 
pilot  is  not  allowed  to  make  changes.  When  the  RAT 
is  activated,  this  point  always  appears  on  the  route  line 
as  a  green  dot,  and  the  route  line  changes  to  the  color 
green  after  this  point  in  time/space.  In  other  words,  the 
color-coding  suggests  that  the  pilot  may  make  changes 
at  any  point  beyond  the  green  dot.  If  a  waypoint  ap¬ 
pears  below  this  null  point,  it  is  represented  in  pink  to 
suggest  that  changes  cannot  be  made  at  that  waypoint. 
If  a  waypoint  appears  above  the  null  point,  it  is  outlined 
in  green,  and  its  label  is  displayed  in  green.  Because  the 
field  (box)  adjacent  to  the  label  “Name”  is  empty  and 
there  is  an  orange  circle  around  the  null  point,  the  pilot 
would  know  the  null  point  is  currendy  selected.  Therefore, 
the  window  in  Figure  A 4  represents  a  case  in  which  the 
aircraft  s  current  heading  is  328  degrees  and  the  heading 
at  the  null  point  will  change  to  336  degrees. 


To  make  an  altitude  change  (at  a  waypoint  beyond 
the  null  point),  the  pilot  again  would  select  the  arrow 
pointing  right  (>).  This  arrow  allows  the  pilot  to  select 
any  of  the  waypoints  that  are  currently  visible  on  the 
display  but  are  beyond  the  null  point.  For  example,  if  the 
CDTI  appeared  as  in  Figure  A 4,  selecting  the  right  arrow 
once  would  result  in  the  window  appearing  as  it  does  in 
Figure  A5.  If  the  pilot  selected  waypoint  OAL  -099  (as 
in  Figure  A5),  a  corresponding,  orange  circle  would  ap¬ 
pear  around  the  star  representing  waypoint  OAL  —099. 
The  window  in  Figure  A5  would  inform  the  pilot  that 
the  current  plan  of  action  is  to  head  into  OAL  -099  at 
336  degrees  and  change  heading  to  335  degrees  at  that 
waypoint.  In  addition,  the  pilot  would  know  that  the 
planned  altitude  at  that  waypoint  is  40,000  feet. 

The  pilot  has  the  option  of  selecting  any  point  along 
the  route  line,  even  if  it  is  not  pre-defined  as  a  waypoint. 
To  do  so,  the  pilot  places  the  mouse  (which  is  the  instru¬ 
ment  for  interface  in  this  simulation)  over  the  route  line. 
A  large  arrow  appears,  to  acknowledge  that  a  point  may 
be  selected.  Thereafter,  the  pilot  clicks  the  display  at  the 
desired  point.  Consequently,  the  new  waypoint  appears 
on  the  display  and  is  labeled.  This  new  waypoint  becomes 
an  option  when  toggling  through  the  name  field  with  the 
right  (>)  and  left  (<)  arrows. 

To  change  altitude  at  any  pre-defined  or  customized 
waypoint,  the  pilot  first  must  select  the  waypoint  (via  the 
menu).  Once  the  waypoint  is  selected,  the  pilot  must  click 
the  field  labeled  “Alt.”  As  shown  in  Figure  A6,  the  fields 
corresponding  to  altitude  are  highlighted  in  the  original 
window,  and  a  scroll  bar  appears.  The  pilot  is  allowed 
to  assess  changes  in  altitude  by  manipulating  either  of 
the  two  windows.  Both  the  arrows  (Vand  A)  on  the 
original  window  to  the  right  and  the  white  arrows  on  the 
scroll  bar  window  («  and  »)  to  the  left  allow  the  pilot 
to  assess  changes  in  altitude  in  intervals  of  1,000  feet. 
The  remaining  white  arrows  on  the  scroll  bar  window 
(<  and  >)  to  the  left  allow  the  pilot  to  assess  changes  in 
intervals  of  100  feet.  The  pilot  also  is  able  to  select  the 
pointer  and  slide  it  along  the  scroll  bar  to  obtain  any 
desired  value. 

When  a  new  altitude  is  proposed,  the  CDTI  changes 
appearance  in  several  ways.  Figure  A7  provides  an  example 
of  these  changes.  In  this  situation,  the  aircraft  currently 
is  at  40,000  feet,  but  an  altitude  of  43,000  feet  is  being 
considered.  The  scroll  bar  represents  these  changes  by 
presenting  the  proposed  altitude  in  both  absolute  and 
relative  terms  (e.g.,  43,000  (+3,000) ).  Similarly,  the  “Alt 
s/c”  field  represents  the  altitude  at  which  the  aircraft  will 
start  its  climb  (i.e.,  40,000  feet),  and  the  “Alt  End”  field 
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Figure  A4.  NASA’s  CDTi  with  the  RAT  tool  and  its  respective  window  activated  point”  is 
selected. 


OAL-099 


14000  Cl 


Figure  AS.  An  example  of  the  window  when  a 
waypoint  is  selected. 


Figure  A6.  An  example  of  the  windows  that  are  manipulated  to  assess  changes  in  altitude. 
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indicates  the  altitude  at  which  the  aircraft:  will  end  (i.e., 
fly  level  at  43,000  feet). 

The  upper  portion  of  the  display  also  changes  accord¬ 
ingly.  A  green  diamond  appears  on  the  route  line,  and 
it  is  labeled  S/C.  This  diamond  represents  the  point 
at  which  the  aircraft  must  begin  its  ascent.  The  point  at 
which  the  desired  altitude  will  be  reached  (in  this  case 
OAL  -099)  is  labeled  with  the  desired  altitude  (i.e.,  FL 
430).  The  portion  of  the  route  over  which  the  aircraft 
will  climb  is  colored  in  pink. 

The  V-Spd  field  presents  the  optimal  vertical  speed 
for  the  proposed  climb  (e.g.,  1 1 50  ft/min).  Like  proposed 
altitudes,  variations  in  vertical  speed  can  be  examined. 
When  V-Spd  is  selected,  the  arrows  on  the  menu  can  be 
used  to  toggle  through  potential  vertical  speeds  at  one- 
hundred-foot  intervals  or  the  scroll  bar  window  may  be 
used  to  examine  100-foot  intervals,  10-foot  intervals,  or 
any  value  so  desired  (via  the  scroll  arrow).  However,  one 


current  drawback  of  the  simulation  is  that  it  does  not  put 
any  limits  on  vertical  speed  (S.  Holland,  personal  com¬ 
munication,  May,  1999).  That  is,  the  CDTI  simulation 
has  not  been  programmed  to  take  the  limitations  of  an 
aircraft  into  account. 

Using  the  RAT  to  assess  altitude  changes  is  helpful 
in  couple  ofways.  First,  it  allows  the  pilot  to  visualize  the 
space  over  which  the  proposed  change  will  take  place.  For 
example,  the  diamond  in  Figure  A7  (which  is  colored 
green  in  the  actual  display)  would  move  farther  down  the 
screen  (i.e.,  closer  to  ownship)  if  the  pilot  was  interested 
in  reaching  45, 000  feet  (vs.  43,000)  at  OAL -099.  On  the 
other  hand,  the  diamond  would  move  farther  up  the  screen 
(i.e.,  closer  to  OAL  —099)  if  the  pilot  was  interested  in 
reaching  4 1 ,000  feet  at  OAL-099.  Furthermore,  decreas¬ 
ing  the  vertical  speed  from  1 150  to  1250  ft/min  would 
cause  the  diamond  to  move  farther  down  the  route  line 
(i.e.,  closer  to  ownship),  whereas  increasing  vertical  speed 
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from  1 150  to  1350  ft/min  would  cause  the  diamond  to 
move  farther  up  the  route  line  (i.e,,  closer  to  OAL-099). 
A  second,  and  probably  more  important,  benefit  gained 
from  the  RAT  is  in  terms  of  conflict  alerting.  If  there  is 
a  possibility  that  the  proposed  change  will  result  in  an 
alert  zone  contact,  ownship  and  the  appropriate  intruder 
aircraft  are  coded  yellow  in  the  actual  display. 

If  an  altitude  and  /  or  vertical  speed  change  is  assessed 
and  the  pilot  finds  it  unacceptable,  three  courses  of  action 
are  possible.  The  pilot  can  select  the  Undo  button,  and 
the  values  are  set  to  their  original  states.  On  the  other 
hand,  the  pilot  can  simply  edit  the  proposed  values.  For 
example,  the  pilot  may  want  to  evaluate  the  effects  of  a 
descent  after  finding  the  effects  of  ascent  unacceptable. 
(Proposing  a  descent  is  much  the  same  as  proposing  an 
ascent.  The  only  difference  is  that  the  label  ctS/C”  is 
replaced  with  “T/D”  to  signify  top  of  descent.)  Finally, 
the  pilot  can  select  the  RAT  button.  In  which  case,  the 


values  are  set  to  their  original  state,  and  the  RAT  tool  is 
deactivated. 

If  the  pilot  assesses  the  effects  of  an  altitude  change 
and  would  like  those  changes  to  occur,  the  Enter  button 
is  selected.  When  this  button  is  selected,  a  small  menu 
appears.  Thereafter,  the  pilot  has  another  chance  to  negate 
the  changes  by  selecting  “Reject.”  If  the  pilot  would  like 
the  proposed  changes  to  occur,  the  “Accept”  option  is 
selected.  Once  the  proposed  changes  are  accepted  by  the 
pilot,  the  display  is  altered.  An  example  of  the  alteration 
can  be  seen  in  Figure  AS.  As  can  be  seen  in  Figure  A8, 
a  dotted  line  (which  is  multicolored  in  pink  and  blue 
in  the  actual  display)  is  used  to  represent  the  area  over 
which  the  climb  will  occur.  Blue  color-coding  is  used  to 
represent  the  area  over  which  the  aircraft  will  fly  at  the 
newly  accepted  flight  level.  (In  cases  of  descent,  blue  is 
replaced  by  brown  color-coding.) 
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Figure  A8.  The  CDTI  as  it  appears  when  a  new  value  for  attitude  is  accepted. 
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When  assessing  and  accepting  changes  in  heading,  the 
CDTI  functions  in  much  the  same  way  as  it  does  with 
altitude  changes.  The  RAT  tool  first  must  be  activated. 
Thereafter,  a  pre-existing  waypoint  or  a  customized  way- 
point  must  be  selected  with  the  mouse.  For  example,  the 
pilot  might  select  waypoint  OAL  -198,  and  the  data 
associated  with  that  point  might  appear  (Figure  A9). 
To  make  changes  in  heading,  the  selected  waypoint  is 
“dragged”  to  another  location  on  the  screen.  Figure  A10 
presents  a  case  where  a  selected  waypoint,  OAL  -0198, 
has  been  moved  to  create  a  new  waypoint,  OAL  -214. 
Notice  the  original  waypoint,  OAL  198,  remains  intact. 
However,  a  new  waypoint  appears  on  the  display.  The 
data  for  this  new  waypoint  appear  in  the  window,  and  it 
reflects  the  proposed  route.  In  this  example,  the  aircraft 
will  approach  waypoint  OAL  -214  at  a  heading  of  25° 


Figure  A9.  Data  fora  sample  waypointto  be  used 
in  making  changes  to  planned  heading. 


and  change  to  311°  when  OAL  —214  is  reached.  If  the 
original  route  is  used  (Figure  A9),  the  aircraft  will  head 
to  waypoint  OAL  198  at  335°  and  change  to  334°  when 
that  waypoint  is  reached. 

Therefore,  the  pilot  is  able  to  assess  both  the  proposed 
route  and  its  data  before  accepting  the  change.  As  with 
assessing  altitude  changes,  the  pilot  also  is  allowed  to  assess 
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Figure  A10.  The  CDTI  as  it  appears  when  changes  to  planned  heading  are  assessed. 
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whether  a  separation  violation  will  occur  as  a  result  of  Although  the  important  features  of  the  simulation 
the  proposed  changes.  To  accept  or  reject  the  proposed  were  summarized  above,  the  preceding  summary  is  not 

heading  changes,  the  pilot  follows  the  same  procedures  complete.  In  fact,  pilots  complete  five  to  six  hours  of 

as  those  described  above.  If  the  pilot  were  to  accept  the  training  before  participating  in  research  at  NASA  (S. 

changes  proposed  in  Figure  A10,  the  CDTI  would  ap-  Holland,  personal  communication.  May,  1999).  The 

pear  as  in  Figure  All.  interested  reader  should  consult  supplemental  sources 

There  are  two  important  aspects  of  the  RAT  tool  that  for  a  complete  understanding  of  the  work  at  NASA  (e.g., 

require  further  comments.  First,  as  a  precautionary  mea-  Cashion  et  al.,  1997;  Johnson  et  al.,  1 997;  Mackintosh 
sure,  the  RAT  tool  automatically  deactivates  if  a  threat  et  al.,  1998). 
is  detected  in  real-time.  Second,  one  apparent  drawback 
of  the  RAT  is  that  it  does  not  allow  the  pilot  to  assess 
changes  in  ground  speed  or  true  airspeed. 


Figure  All.  The  CDTI  as  it  appears  when  changes  to  heading  have  been  accepted. 
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9.0  APPENDIX  B 


Appendix  B  contains  Riley’s  definition  (1993)  of  all  nodes  in  the  general  model  of  information  flow.  If  a  node 
appears  m  gray,  it  was  not  included  in  the  current  model  and  a  justification  is  provided  for  its  exclusion.  Each 
node  included  m  the  current  model  is  accompanied  by  the  question  used  to  identify  characteristics  as  well  as  a  list 


9.1  The  World 


1)  World-.  “Because  the  World  node  represents  everything  outside  your  particular  system,  parameters  of  the  larger 
system  that  your  system  fits  into  should  be  included  in  this  node”  (Riley  1993  p  28)  ® 

3  NASACDTI)?^  affeCtS  ^  traffiC  SkUati0n  m  "  freC'flight  environment>  outside  of  the  pilot  and  the 

b)  Characteristics: 
i)  Ownship 

(1)  State  (i.e.,  position,  speed,  heading,  .vertical  speed,  altitude,  attitude) 

(2)  Destination  (immediate) 

(3)  Limitations/capabilities  of 

•  /-x  AState  °fot,ler  systems  (t*e.,  systems  other  than  the  one  being  examined  here) 

ti)  Other  Aircraft 


(1)  Amount  of  (i.e.,  density  of  traffic) 

(2)  State  (i.e.,  position,  speed,  heading,  vertical  speed,  altitude,  attitude) 

(3)  Destination  (immediate) 

(4)  Limitations/capabilities  of 

...  ^  gtff  °f  °th“  systems  (i.e.,  systems  other  than  the  one  being  examined  here) 
D)  Available  Technology  (i.e.,  whether  they  also  are  equipped  with  a  CDTI) 

iii)  Other  Operators 

(1)  Other  Pilots 

(a)  Levels  of  SA 

(b)  Amount  of  mental  workload 

(2)  Air  Traffic  Managers 

(a)  Levels  of  SA 

(b)  Level  of  mental  workload 

iv)  Environmental  Conditions 

(1)  Weather 

(2)  Wind 

(3)  Turbulence 

(4)  Time  of  da y 

(5)  Terrain 

(6)  Restrictions  due  to  traffic  and  facilities  (e.g.,  airport  closings  or  overuse) 


9.2  THE  HUMAN  SIDE 

9.2.1  Human  Input 

V1th™l^\rld\ Tf  ^  uPreSfntS  ^  ?perat°r’S  aCCCSS  to  mformation  about  the  operational  environment 
through  all  methods  other  than  through  the  system  being  modeled”  (Riley  1993,  p  34) 

a)  Our  question:  What  affects  the  pilot’s  ability  to  perceive  characteristics  of  the  traffic  situation  (i.e.,  informa- 
non  other  than  that  presented  by  the  CDTI)? 

b)  Characteristics: 

i)  Out-the-window 


(1) Time  spent  viewing  out-the-window  information 

(2)  Amount  of  information  available  to  the  pilot 
ii)  Other  instruments 

(1)  Time  spent  reading  other  instruments 

(2)  Readability  of  displays  outside  of  the  system  (e.g.,  glare,  color  of  symbols/text,  size  of  symbols/text) 
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iii)  Accuracy  of  pilots  previous  world  model  (i.e.,  previous  SA  regarding  the  world) 

iv)  Level  of  pilot  mental  workload 

v)  Physical  state  of  pilot  (e.g.,  vision) 

vi)  Amount  of  noise  in  cockpit  environment  (e.g.,  may  affect  ability  to  hear  radio  exchanges) 

vii)  General  piloting  skills  (e.g.,  experience,  ability,  training) 

viii)  Adequacy  of  physical  feedback  from  the  aircraft  (e.g.,  can  pilot  perceive  changes  in  aircraft  altitude, 
etc.?) 

2)  Perceive  Machine  Behavior.  “This  node  refers  to  the  operators  ability  to  sense  what  action  the  system  is  currently 
performing”  (Riley,  1993,  p.  34). 

NOT  INCLUDED-  The  CDTI  does  not  perform  any  control  actions. 

3)  Perceive  Displays'.  “This  node  refers  to  the  operators  act  of  reading  the  displays”  (Riley,  1993,  p.  34). 

a)  Our  question:  What  affects  the  pilots  ability  to  perceive  information  from  the  CDTI? 

b)  Characteristics: 

i)  Pilot  skill  with  the  CDTI  (e.g.,  experience,  ability,  training) 

ii)  Time  spent  reading  displays  in  the  system 

iii)  Time  spent  reading  other  instruments 

iv)  Readability  of  displays  in  the  system  (i.e.,  the  CDTI  and  the  FMS) 

v)  Amount  of  noise  in  cockpit  environment  (e.g.,  may  affect  ability  to  hear  auditory  warnings  from  the 
CDTI) 

vi)  Accuracy  of  current  machine  model  (i.e.,  current  SA  regarding  the  CDTI) 

4) Infer  World  State'.  “This  node  refers  to  the  operators  process  of  making  sense  out  of  the  situation  and  gaining  or 
maintaining  situation  awareness”  (Riley,  1993,  p.  34). 

a)  Our  question:  What  affects  the  processes  involved  in  the  pilot’s  gaining/maintaining  awareness  of  the  traffic 
situation  (outside  of  perceptual  issues)? 

b)  Characteristics: 

i)  Level  of  pilot  mental  workload 

ii)  General  piloting  skills  (e.g.,  experience,  ability,  training) 

iii)  Accuracy  of  pilots  previous  world  model  (i.e.,  previous  SA  regarding  the  world) 

iv)  Number  of  errors  in  perceiving  the  world 

v)  Level  of  confidence  in  perception  of  the  world 

vi)  Accuracy  of  current  machine  model  (i.e.,  current  SA  regarding  the  CDTI) 

5)  Infer  Machine  State:  “This  node  refers  to  the  operators  process  of  understanding  what  the  machine  is  doing” 
(Riley,  1993,  p.  34). 

a)  Our  question:  What  affects  the  pilots  process  of  understanding  of  what  the  CDTI  is  doing? 

b)  Characteristics: 

i)  Level  of  pilot  mental  workload 

ii)  Pilot  skill  with  the  CDTI  (e.g.,  experience,  ability,  training) 

iii)  Number  of  errors  in  perceiving  the  CDTI 

iv)  Level  of  confidence  in  perception  of  the  CDTI 

v)  Level  of  confidence  in  the  CDTI  (i.e.,  can  I  trust  the  machine?) 

vi)  Accuracy  of  pilot’s  current  world  model  (i.e.,  current  SA  regarding  the  world) 

vii)  Accuracy  of  pilot’s  previous  machine  model  (i.e.,  previous  SA  regarding  the  CDTI) 

6)  World  Model:  “This  represents  the  operators  level  of  understanding  about  the  operational  environment”  (Riley, 
1993,  p.  35). 

a)  Our  question:  What  affects  the  pilot’s  current  mental  model  of  the  traffic  situation  or  what  affects  the  opera 
tor’s  awareness  of  the  traffic  situation? 

b)  Characteristics 
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i)  Accuracy  of  pilots  current  world  model  (i.e.,  current  SA  regarding  the  world)  (This  characteristic  is  merely  a 
place  holder,  because  most  characteristics  were  identified  under  “Infer  World  State.”) 

7)  Machine  Model :  “This  represents  the  operator  s  level  of  understanding  about  the  system,  such  as  its  current  level 
of  reliability 5  (Riley,  1993,  p.  35). 

a)  Our  question:  What  affects  the  pilots  current  mental  model  of  the  CDTI? 

b)  Characteristics 

i)  Accuracy^  of  current  machine  model  (I.e.,  current  SA  regarding  the  CDTI)  (This  characteristic  is  merely 
a  place  holder,  because  most  characteristics  were  identified  under  “Infer  Machine  State.”) 

8)  Machine's  Goals :  “This  represents  the  operators  understanding  of  the  machine  s  current  goals  and  targets”  (Riley, 
1993,  p.  35). 

a)  Our  question:  What  affects  the  pilots  ability  to  understand  the  current  “goals”  of  the  CDTI? 

b)  Characteristics 

i)  Pilot  skill  with  the  CDTI  (e.g.,  experience,  ability,  training) 

ii)  Level  of  confidence  In  the  CDTI  (i.e.,  can  I  trust  the  machine?) 

9)  Predict  Machine  Behavior.  “This  node  refers  to  the  operators  anticipation  of  the  next  actions  to  be  taken  by  the 
machine”  (Riley,  1993,  p.  35). 

a)  Our  question:  What  affects  the  pilots  ability  to  anticipate  the  warnings  given  by  the  CDTI? 

b)  Characteristics 

I)  Accuracy  of  pilots  current  world  model  (i.e.,  current  SA  regarding  the  world) 
ii)  Accuracy  of  current  machine  model  (i.e.,  current  SA  regarding  the  CDTI) 

9.2.2  Human  Output 

1) Plan  Own  Action :  “This  refers  to  the  operators  process  of  deciding  what  to  do  next”  (Riley,  1993,  p.  35). 

a)  Our  question:  What  affects  the  appropriateness  of  the  action  chosen  by  the  pilot? 

b)  Characteristics: 

i)  Accuracy  of  pilots  current  world  model  (i.e.,  current  SA  regarding  the  world) 

ii)  Level  of  confidence  in  pilot  s  current  world  model  (i.e.,  in  current  SA  regarding  the  world) 

iii)  Accuracy  of  current  machine  model  (i.e.,  current  SA  regarding  the  CDTI) 

iv)  Level  of  confidence  In  current  machine  model  (i.e.,  in  current  SA  regarding  the  CDTI) 

v)  Accuracy  of  self  model 

vi)  Accuracy  In  predicting  the  machine  s  behavior 

vii)  Perceived  time  pressure 

viii)  Current  team  SA 

ix)  Appropriateness  of  operator  s  goals 

x)  Level  of  confidence  in  the  CDTI  (i.e.,  tan  I  trust  the  machine?) 

xi)  Level  of  confidence  in  position  sensors 

2)  Operators  Goals:  “This  node  represents  the  operator  s  actual  intentions,  and  can  contain  characteristics  that 
are  the  operators  counterparts  of  the  Operators  Goals  and  Machines  Goals  nodes  on  the  Machine  side  of  the 
model”  (Riley,  1993,  p.  35). 

a)  Our  question:  What  affects  the  intentions  of  the  pilot? 

b)  Characteristics: 

i)  Accuracy  of  pilots  previous  world  model  (i.e.,  previous  SA  regarding  the  world) 

ii)  Accuracy  of  pre-flight  planning 

3)  Self  Model:  “This  refers  to  the  operators  assessment  of  his  or  her  own  current  abilities  or  state”  (Riley,  1993,  p. 

35). 

a)  Our  question:  What  affects  the  pilot s  ability  to  assess  his/her  abilities  to  deal  with  the  traffic  situation  (phys 
ical  and/or  mental)? 

b)  Characteristics: 

I)  Level  of  trust  In  automated  systems  performing  control  actions 
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ii)  Level  of  trust  in  the  CDTI 

iii)  General  piloting  skills  (e.g.,  experience,  ability,  training) 

4)  Take  No  Action  or  Monitor,  “This  is  basically  a  place  holder  for  one  of  the  possible  actions  of  the  operator”  (Ri¬ 
ley,  1993,  p.  36). 

a)  Our  question:  What  affects  the  pilot  in  accurately  monitoring  the  CDTI? 

b)  Characteristics: 

i)  Appropriateness  of  planned  action 

5)  Provide  Information :  “This  node  represents  the  process  by  which  the  operator  enters  data  into  the  system  or  pro¬ 
vides  other  types  of  information”  (Riley,  1993,  p.  36). 

a)  Our  question:  What  affects  the  pilot  in  accurately  providing  data  to  the  CDTI  or  the  FMS? 

b)  Characteristics: 

i)  Appropriateness  of  planned  action 

ii)  Accuracy  of  the  pilot  in  providing  information  to  the  CDTI  or  FMS  (e.g.,  via  keystroke  errors  or  the 
pilot  possesses  inaccurate  information) 

6)  Request  Information'.  “This  node  represents  the  process  by  which  the  operator  enters  requests  for  information, 
such  as  calling  up  new  display  pages”  (Riley,  1993,  p.  36). 

a)  Our  question:  What  affects  the  pilot  in  requesting  information  from  the  CDTI  (e.g.,  a  prediction  regarding 
the  flight  path  of  another  aircraft)? 

b)  Characteristics: 

i)  Appropriateness  of  planned  action 

ii)  Accuracy  of  the  pilot  in  providing  information  to  the  CDTI  (e.g.,  via  keystroke  errors  or  the  pilot  pos¬ 
sesses  inaccurate  information) 

iii)  Level  of  confidence  in  planned  action  (i.e.,  does  the  pilot  need  more  information?) 

7)  Command:  “This  node  represents  inputs  made  by  the  operators  to  change  the  state  of  the  system  through  a 
command  to  automation”  (Riley,  1993,  p.  36). 

a)  Our  question:  What  affects  the  pilot  in  commanding  the  CDTI  to  change  its  state  (e.g.,  changing  the  dis 
play  options  like  viewing  of  the  datatags)? 

b)  Characteristics: 

i)  Appropriateness  of  planned  action 

ii)  Accuracy  of  the  pilot  in  providing  information  to  the  CDTI  (e.g.,  via  keystroke  errors  or  the  pilot  pos 
sesses  inaccurate  information) 

iii)  Level  of  confidence  in  planned  action  (i.e.,  does  the  pilot  need  more  information?) 

8)  Control:  “This  refers  to  inputs  made  by  the  operators  through  manual  control”  (Riley,  1993,  p.  36). 

a)  Our  question:  What  affects  the  pilot  in  controlling  the  aircraft’s  position? 

b)  Characteristics: 

i)  Appropriateness  of  planned  action 

ii)  Accuracy  of  physical  control 

(1)  Adequacy  of  physical  feedback  from  the  aircraft  (e.g.,  can  pilot  perceive  changes  in  aircraft  altitude, 
etc.?) 

(2)  Adequacy  of  physical  feedback  from  the  control  device  (e.g.,  force) 

iii)  Level  of  confidence  in  planned  action  (i.e.,  does  the  pilot  need  more  information?) 
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93  THE  MACHINE  SIDE 


9.3.1  Machine  Input 

1)  World  Sensors:  “This  node  is  intended  to  contain  all  the  potential  sources  of  information  coming  into  the  sys¬ 
tem”  (Riley,  1993,  p.  29)* 

a)  Our  question:  What  affects  the  information  received  by  the  world  sensors,  or  what  affects  the  quality/accuracy 
of  this  information? 
b)  Characteristics: 

i)  Resolution  and  update  rate  of  position  sensors 

ii)  State  of  position  sensors  hardware  (Le.,  functioning  or  malfunctioning) 

iii)  Weather . 

2)  Control  Sensors:  This  node  is  meant  to  represent  all  the  ways  the  operators  can  put  information  into  the  system, 
typically  through  controls”  (Riley,  1993,  p,  29). 

a)  Our  question:  What  affects  the  information  received  by  the  control  sensors,  or  what  affects  the  quality/ 
accuracy  of  this  information? 

b)  Characteristics: 

i)  Type  of  action  chosen  by  pilot 

ii)  Resolution  and  update  rate  of  control  sensors 

iii)  State  of  control  sensors  hardware  (Le.,  functioning  or  malfunctioning)  for  all  forms  of  input  (control 
devices  and  keypad  entries) 

3)  Operator  Sensors:  This  node  is  relevant  for  systems  that  include  sensors  to  detect  the  operator  s  cognitive  or 
physiological  state”  (Riley,  1993,  p.  29)* 

NOT  INCLUDED-  The  CDTI  does  not  monitor  the  state  of  the  pilot. 

4) Machines  Goals:  This  node  represents  the  machines  current  targets,  operational  parameters,  or  understanding 
of  the  current  mission  goals”  (Riley,  1993,  p.  29). 

a)  Our  question:  What  affects  the 4  ability”  of  the  CDTI  to  4  understand”  its  role? 

b)  Characteristics: 

i)  Accuracy  of  machines  goals  (e.g.,  did  the  programmers  model  match  the  users  model?) 

5)  Infer  Operator  State,  This  node  represents  functions  that  can  use  the  information  generated  by  the  Operator 
Sensors  node  and  infer  what  the  operators  current  cognitive  or  physiological  state  is”  (Riley,  1993,  p.  30). 

NOT  INCLUDED-  The  CDTI  does  not  monitor  the  state  of  the  pilot. 

6)  Operators  Goals:  This  node  represents  the  machine  s  understanding  of  the  operator  s  current  goals.  It  is  used  in 
models  of  systems  that  can  infer  the  operators  intentions  based  pardy  on  this  understanding”  (Riley,  1993,  p. 
29). 

a)  Our  question:  What  affects  the  ability”  of  the  CDTI  to  “understand”  the  operator  s  current  and  specific 
goals  (e.g.,  what  affects  the  ability  of  the  CDTI  to  understand  current  heading,  intended  destination,  etc,)? 

b)  Characteristics: 

i)  Accuracy  of  the  pilot  in  providing  information  to  the  CDTI  (e.g.,  via  keystroke  errors  or  the  pilot  pos 
sesses  inaccurate  information) 

ii)  State  (Le.,  functioning  or  malfunctioning)  of  the  systems  hardware/software  (Le.,  the  CDTI  and  FMS 
hardware/software  and  the  software  that  allows  them  to  “communicate) 

7)  Infer  Operator  Intent,  This  node  represents  the  machines  process  of  inferring  what  the  operator  s  intentions  are. 
This  is  relevant  for  the  Operator  Intent  Responsive  level  of  intelligence  and  up...  ”  (Riley,  1993,  p.  30). 

NOT  INCLUDED-  The  CDTI  does  not  infer  intent  and  respond. 
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8)  Infer  World  State:  “This  node  represents  the  machine  s  process  of  understanding  what  the  current  state  of  the  op¬ 
erational  environment  is,  based  on  the  information  provided  to  it  from  the  World  Sensors”  (Riley,  1993,  p.  29). 

a)  Our  question:  What  affects  the  processes  involved  in  the  CDTI  gaining/maintaining  an  “understanding”  of 
the  traffic  situation? 

b)  Characteristics: 

i)  Accuracy  of  information  the  CDTI  has  regarding  operator’s  goals 

ii)  Accuracy  of  machine  s  goals 

iii)  State  of  position  sensors  hardware  (i.e.,  functioning  or  malfunctioning) 

iv)  State  (i.e.,  functioning  or  malfunctioning)  of  the  systems  hardware/software  (i.e.,  the  CDTI  and  FMS 
hardware/software  and  the  software  that  allows  them  to  “communicate) 

v)  State  of  position  sensors  hardware  (i.e.,  functioning  or  malfunctioning) 

9)  Operator  Models :  “This  node  represents  internal  representations  of  the  human  operators  used  by  the  machine  to 
draw  inferences  about  the  operators  intention  and  possible  future  actions.  Again,  no  such  capabilities  are  cur¬ 
rently  provided  in  commercial  transport  equipment ...  This  node  would  be  relevant  to  a  system  that  can  merely 
be  personalized  for  a  particular  operator”  (Riley,  1993,  p.  29). 

a)  Our  question:  What  affects  the  ability  of  the  CDTI  to  “understand”  what  the  operator  wants  personalized? 

b)  Characteristics: 

i)  Accuracy  of  the  pilot  in  providing  information  to  the  CDTI  (e.g.,  via  keystroke  errors  or  the  pilot  pos 
sesses  inaccurate  information) 

ii)  State  of  control  sensors  hardware  (i.e.,  functioning  or  malfunctioning)  for  all  forms  of  input  (control 
devices  and  keypad  entries) 

iii)  State  (i.e.,  functioning  or  malfunctioning)  of  the  system’s  hardware/software  (i.e.,  the  CDTI  and  FMS 
hardware/software  and  the  software  that  allows  them  to  “communicate) 

10)  Infer  Operators  Knowledge*.  “This  node  is  intended  to  represent  the  machine’s  process  of  inferring  the  opera¬ 
tor’s  knowledge  about  the  situation”  (Riley,  1993,  p.  31). 

NOT  INCLUDED-  The  CDTI  does  not  infer  what  the  pilot  does/does  not  know. 

11)  World  Model:  “This  node  represents  the  machines  internal  representation  of  the  operational  environment” 
(Riley,  1993,  p.  29). 

a)  Our  question:  What  affects  the ’CDTI  s  current  model  of  the  traffic  situation? 

b)  Characteristics: 

i)  Accuracy  of  the  machine  s  world  model  (This  characteristic  is  merely  a  place  holder,  because  most  char 

acteristics  were  identified  under  “Infer  World  State.”) 

121 _ Predict  Operators  Behavior.  “This  node  is  relevant  for  very  complex  systems  that  can  not  only  infer  the 

operator  s  intentions  and  cognitive  and  physiological  state  but  can  also  predict  potential  errors  the  operator  may 
make,  or  other  actions  so  it  can  assist  with  those  actions”  (Riley,  1993,  p.  31). 

NOT  INCLUDED-  Although  the  RAT  tool  assists  the  operator  in  assessing  what  should  be  done  in  the  future, 
the  CDTI  does  not  autonomously  predict  an  operators  behavior. 

9.3.2  Machine  Output 

1)  Determine  Operators  Need for  Information:  “This  node  contains  different  items  depending  on  the  overall  levels 
of  capability  of  the  system...”  (Riley,  1993,  p.  31). 

NOT  INCLUDED-  The  CDTI  does  not  decide  (for  the  operator)  what  information  should  be  displayed. 

2)  Plan  Own  Action:  “This  node  represents  the  machine’s  process  of  planning  its  own  future  actions”  (Riley,  1993, 
p.  32). 

NOT  INCLUDED-  The  CDTI  does  not  plan  its  own  actions,  because  it  does  not  make  any  control  actions. 

3)  Check  Permission:  “This  node  refers  to  the  machine’s  checking  whether  or  not  the  operator  has  granted  it  per- 
'  mission  to  take  over  operator  tasks  when  it  determines  that  the  operator  needs  help”  (Riley,  1993,  p.  32). 

NOT  INCLUDED-  The  CDTI  does  not  make  control  actions. 
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4)  Request  Permission :  “This  node  refers  to  the  machine  s  process  of  requesting  permission  from  the  operator  to  take 
over  a  task  which  might  benefit  from  automation  but  for  which  the  operator  has  not  granted  standing  permis¬ 
sion  so  the  automation  can  take  over 5  (Riley,  1993,  p.  32). 

NOT  INCLUDED-  The  CDTI  does  not  make  control  actions. 

5)  Request  Information'.  “This  node  refers  to  the  machines  process  of  requesting  information  from  the  operator  that 
the  machine  would  not  otherwise  have  available55  (Riley,  1993,  p.  32). 

NOT  INCLUDED-  The  CDTI  does  not  request  information. 

6)  Provide  Decision :  “This  represents  the  decisions  made  by  the  machine  to  the  human  operator55  (Riley,  1993,  p. 
32). 

a)  Our  question:  What  affects  the  CDTIs  current  model  of  the  traffic  situation? 

b)  Characteristics:  What  affects  the  CDTIs  ability  to  identify  other  aircraft  that  are  likely  to  have  an  alert  zone 
contact  with  ownship? 

i)  Accuracy  of  machines  goals  (e.g.,  the  adequacy  of  the  alerting  logic  used  in  programming) 

7)  Machine's  Goals'.  “This  node  is  the  same  as  the  previous  Machines  Goals  nodes.  Its  characteristics  need  not  be 
repeated  because  they  have  already  been  entered  into  the  list  from  the  previous  one15  (Riley,  1993,  p.  32). 

8)  Self Modek  “This  node  represents  the  machines  knowledge  of  its  own  operational  state55  (Riley,  1993,  p.  33). 
NOT  INCLUDED-  To  our  knowledge,  the  CDTI  does  not  have  its  own  built-in  test. 

9)  Prioritize  Information :  “This  node  represents  the  process  of  prioritizing  all  the  information  awaiting  display  to 
the  operator  and  putting  it  into  a  queue  for  assignment  to  display  devices55  (Riley,  1993,  p.  33). 

NOT  INCLUDED-  The  CDTI  presents  all  information  it  has  available. 

10)  Construct  Displays'.  “This  node  refers  to  the  process  of  taking  the  information  to  be  sent  to  the  crew  and  gen¬ 
erating  the  displays  required  to  do  so.  ..it  merely  represents  the  process  of  converting  the  information  available 
to  the  system  into  display  formats  for  the  crew55  (Riley,  1993,  p.  33). 

a)  Our  question:  What  affects  the  “ability55  of  the  CDTI  to  process  the  information? 

b)  Characteristics: 

i)  Accuracy  of  the  machines  “operators  model” 

ii)  Accuracy  of  the  machines  world  model 
lii)  CDTIs  processing  time 

1 1)  Cache :  “This  represents  a  memory  store  where  information  that  cannot  be  displayed  to  the  crew  due  to  dis 
play  limitations  waits  to  be  displayed”  (Riley,  1993,  p.  33). 

NOT  INCLUDED-  The  CDTI  presents  all  information  it  has  available. 

12)  Displays'.  “This  node  refers  to  the  physical  display  devices”  (Riley,  1993,  p.  33). 

a)  Our  question:  What  affects  the  “ability”  of  the  CDTI  to  present  the  information? 

b)  Characteristics: 

i)  Accuracy  in  constructing  the  display 

ii)  Refresh  rate  of  the  CDTI 

13)  Action :  “This  node  refers  to  the  process  of  actually  performing  some  function  that  changes  the  state  of  the 
operational  environment”  (Riley,  1993,  p.  33). 

NOT  INCLUDED-  The- CDTI  does  not  make  any  control  actions. 
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Table  D1.  Relative  Influence  of  Characteristics. 


Characteristic 

Score 

Rank 

Weather 

42 

/ 

General  piloting  skills 

32 

2 

Time  of  dav 

31 

3 

Terrain 

25 

4 

Ovnship  state 

24 

5 

J  xv el  of  pilot  mental  workload 

24 

5 

Perceived  time  pressure 

22 

7 

State  of  other  systems  in  ownship 

20 

8 

'fime  spent  reading  displays  in  the  system 

19 

9 

Amount  of  other  aircraft 

18 

10 

Turbulence 

18 

10 

'lime  spent  viewing out-the-window  information 

18 

10 

State  of  the  system’s  hardware/software 

17 

13 

Accuracy  of  pre-flight  planning 

16 

14 

'1  tme  spent  reading  other  instruments 

16 

14 

Physical  state  of  pilot 

16 

14 

Amount  of  out-the-window  information  available  to  the  pilot 

14 

17 

Pilot  skill  with  the  AMPS  display 

14 

17 

Accuracy  of  machine’s  goals 

14 

17 

Air  traffic  managers’  levels  of  SA 

13 

20 

Restrictions  due  to  traffic  and  facilities 

13 

20 

Adequacy  of  physical  feedback  from  the  aircraft 

13 

20 

State  of  position  sensors  hardware 

13 

20 

Wind 

12 

24 

Readability  of  displays  in  the  system 

12 

24 

Current  team  SA 

12 

24 

1  limitations/ capabilities  of  ownship 

11 

27 

Other  pilots5  levels  of  SA 

11 

27 

Resolution  and  update  rate  of  position  sensors 

11 

27 

Type  of  action  chosen  by  pilot 

11 

27 

Accuracy  of  information  the  A  MRS  display  has  regarding  operator’s  goals 

10 

31 

]  xrvcl  of  air  traffic  managers5  mental  workload 

9 

32 

Readability  of  displays  outside  of  the  system 

9 

32 

Amount  of  noise  in  cockpit  environment 

9 

32 

Accuracy  of  the  machine’s  world  model 

9 

32 

State  of  other  aircraft 

8 

36 

Destination  (immediate)  of  other  aircraft 

8 

36 

J  imitations/ capabilities  of  other  aircraft. 

8 

36 

1  X’vel  of  confidence  in  pilot’s  current  world  model 

8 

36 

Accuracy  in  constructing  the  display 

8 

36 

Accuracy  of  the  machine’s  “operator’s  model” 

8 

36 

Available  technology  in  other  aircraft 

7  ! 

42 

Other  pilots’  amount  of  mental  workload 

7 

42 

Accuracy  of  pilot’s  current  world  model  ★ 

7 

42 

Revel  of  confidence  in  current  machine  model 

7 

42 

I  jcvel  of  confidence  in  position  sensors 

7 

42 

Accuracy  of  self  model 

7 

42 

Resolution  and  update  rate  of  control  sensors 

7 

42 

Ownship  (immediate)  destination 

6 

49 

Number  of  errors  in  perceiving  the  AMI  (S  display 

6 

49 

Accuracy  of  current  machine  model  * 

6 

49 

1  x*vel  of  confidence  in  planned  action 

6 

49 

Accuracy  of  the  pilot  in  providing  information  to  the  AMPS  display 

6 

49 

Adequacy  of  physical  feedback  from  the  control  device 

6 

49 

State  of  control  season;  hardware  for  all  forms  of  input 

6 

49 

State  of  systems  in  other  aircraft 

5 

56 

Accuracy  of  pilot’s  previous  world  model 

5 

56 

1  xrvel  of  confidence  in  perception  of  the  world 

5 

56 

Accuracy  in  predicting  the  machine’s  behavior 

5 

56 

Level  of  confidence  in  automated  systems  performing  control  actions 

5 

56 

AMES  display’s  processing  time 

5 

56 

Refresh  rate  of  the  AMES  display 

5 

56 

Accuracy  of  pilot’s  previous  machine  model 

4 

63 

Level  of  confidence  in  perception  of  the  AMES  display 

4 

63 

Number  of  errors  in  perceiving  the  world 

2 

65 

Appropriateness  of  operator’s  goals 

2 

65 

Accuracy  of  physical  control 

2 

65 

Appropriateness  of  planned  action 

1 

68 

Table  D2.  Relative  Sensitivity  of  Characteristics. 


Characteristic 

Score 

Rank 

Type  of  action  chosen  by  pilot 

53 

1 

Level  of  pilot  mental  workload 

40 

2 

Appropriateness  of  planned  action 

39 

3 

Ownship  state 

33 

4 

Level  of  air  traffic  managers’  mental  workload 

31 

5 

Accuracy  of  current  machine  model  ★ 

29 

6 

Level  of  confidence  in  planned  action 

29 

6 

Time  spent  viewing  out-the-window  information 

26 

8 

Accuracy  in  predicting  the  machine’s  behavior 

25 

9 

Time  spent  reading  other  instruments 

25 

9 

Ownship  (immediate)  destination 

24 

11 

Level  of  confidence  in  current  machine  model 

24 

11 

Time  spent  reading  displays  in  the  system 

24 

11 

Other  pilots’  amount  of  mental  workload 

23 

14 

Accuracy  of  pilot’s  current  world  model  ★ 

22 

15 

Number  of  errors  in  perceiving  the  world 

20 

16 

Number  of  errors  in  perceiving  the  AMES  display 

18 

17 

Level  of  confidence  in  automated  systems  performing  control  actions 

18 

17 

Level  of  confidence  in  pilot’s  current  world  model 

17 

19 

Appropriateness  of  operator’s  goals 

17 

19 

Accuracy  of  the  pilot  in  providing  information  to  the  AMES  display 

16 

21 

Level  of  confidence  in  perception  of  the  world 

15 

22 

Accuracy  of  physical  control 

15 

22 

Perceived  time  pressure 

15 

22 

Current  team  SA 

15 

22 

Other  pilots’  levels  of  SA 

12 

26 

Level  of  confidence  in  perception  of  the  AMES  display 

12 

26 

Accuracy  of  the  machine’s  world  model 

12 

26 

Air  traffic  managers’  levels  of  SA 

11 

29 

Level  of  confidence  in  position  sensors 

10 

30 

Destination  (immediate)  of  other  aircraft 

6 

31 

Accuracy  of  pre-flight  planning 

6 

31 

Accuracy  of  the  machine’s  “operator’s  model” 

6 

31 

Accuracy  of  machine’s  goals 

6 

31 

Restrictions  due  to  traffic  and  facilities 

5 

35 

Amount  of  noise  in  cockpit  environment 

5 

35 

Amount  of  out-the-window  information  available  to  the  pilot 

4 

37 

Readability  of  displays  in  the  system 

4  ! 

37 

Adequacy  of  physical  feedback  from  the  aircraft 

4 

37 

Adequacy  of  physical  feedback  from  the  control  device 

4 

37 

Limitations/capabilities  of  ownship 

3 

41 

Amount  of  other  aircraft 

3 

41 

Accuracy  of  pilot’s  previous  world  model 

3 

41 

Readability  of  displays  outside  of  the  system 

3 

41 

Accuracy  of  self  model 

3 

41 

Physical  state  of  pilot 

3 

41 

Resolution  and  update  rate  of  position  sensors 

3 

41 

State  of  position  sensors  hardware 

3 

41 

Resolution  and  update  rate  of  control  sensors 

3 

41 

State  of  control  sensors  hardware  for  all  forms  of  input 

3 

41 

State  of  other  aircraft 

2 

51 

Weather 

2 

51 

Accuracy  of  pilot’s  previous  machine  model 

2 

51 

Accuracy  of  information  the  AMES  display  has  regarding  operator’s  goals 

2 

51 

AMES  display’s  processing  time 

2 

51 

State  of  other  systems  in  ownship 

1 

56 

State  of  systems  in  other  aircraft 

1 

56 

Wind 

1 

56 

Turbulence 

1 

56 

Pilot  skill  with  the  AMES  display 

1 

56 

Accuracy  in  constructing  the  display 

1 

56 

Refresh  rate  of  the  AMES  display 

1 

56 

Limitations/capabilities  of  other  aircraft 

0 

63 

Available  technology  in  other  aircraft 

0 

63 

Time  of  day 

0 

63 

Terrain 

0 

63 

General  piloting  skills 

0 

63 

State  of  the  system’s  hardware/software 

0 

63 

Table  D3.  Complete  List  of  Tradeoffs. 


Characteristic 

Tradeoff  Characteristic 

Ownship  state 

Level  of  pilot  mental  workload 

Type  of  action  chosen  by  pilot 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  displays  in  the  system 

Time  spent  reading  other  instruments 

Current  team  S  A 

Perceived  time  pressure 

Accuracy  of  pilot's  current  world  model  ★ 

Level  of  confidence  in  position  sensors 

Amount  of  out-the-window  information  available  to  the  pilot 

Number  of  errors  in  perceiving  the  AMES  display 

Accuracy  of  current  machine  model  ★ 

Physical  state  of  pilot 

Level  of  pilot  mental  workload 

Ownship  state 

Type  of  action  chosen  by  pilot 

Level  of  air  traffic  managers'  mental  workload 

Current  team  S  A 

Other  pilots'  amount  of  mental  workload 

Level  of  confidence  in  planned  action 

Level  of  confidence  in  perception  of  the  world 

Level  of  confidence  in  pilot's  current  world  model 

Level  of  confidence  in  current  machine  model 

Level  of  confidence  in  perception  of  the  AMES  display 

Accuracy  in  predicting  the  machine’s  behavior 

Type  of  action  chosen  by  pilot 

Ownship  state 

Level  of  pilot  mental  workload 

Level  of  air  traffic  managers'  mental  workload 

Current  team  SA 

Other  pilots’  amount  of  mental  workload 

Level  of  confidence  in  planned  action 

Perceived  time  pressure 

Level  of  confidence  in  automated  systems  performing  control  actions 

Ownship  (immediate)  destination 

Destination  (immediate)  of  other  aircraft 

Time  spent  viewing  out-the- window  information 

Ownship  state 

Time  spent  reading  displays  in  the  system 

Level  of  air  traffic  managers'  mental  workload 

Time  spent  reading  other  instruments 

Level  of  confidence  in  planned  action 

Level  of  confidence  in  perception  of  the  world 

Level  of  confidence  in  pilot's  current  world  model 

Level  of  confidence  in  automated  systems  performing  control  actions 

Level  of  confidence  in  current  machine  model 

Time  spent  reading  displays  in  the  system 

Ownship  state 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  other  instruments 

Level  of  confidence  in  planned  action 

Level  of  confidence  in  automated  systems  performing  control  actions 

Ownship  (immediate)  destination 

Level  of  confidence  in  current  machine  model 

Level  of  confidence  in  perception  of  the  AMES  display 

Level  of  confidence  in  position  sensors 

Level  of  air  traffic  managers'  mental  workload 

Level  of  pilot  mental  workload 

Type  of  action  chosen  by  pilot 

Time  spent  viewing  out-the-window  information 

Current  team  S  A 

Other  pilots'  amount  of  mental  workload 

Other  pilots'  levels  of  S  A 

Air  traffic  managers'  levels  of  SA 

Time  spent  reading  other  instruments 

Ownship  state 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  displays  in  the  system 

Level  of  confidence  in  planned  action 

Level  of  confidence  in  perception  of  the  world 

Level  of  confidence  in  pilot's  current  world  model 

Level  of  confidence  in  automated  systems  performing  control  actions 

D3 


Table  D3.  Complete  List  of  Tradeoffs  (continued). 


Current  team  SA 

Ownship  state 

Level  of  pilot  mental  workload 

Type  of  action  chosen  by  pilot 

Level  of  air  traffic  managers’  mental  workload 

Perceived  time  pressure 

Accuracy  of  pilot’s  current  world  model  ★ 

Amount  of  noise  in  cockpit  environment 

Other  pilots’  amount  of  mental  workload 

Level  of  pilot  mental  workload 

Type  of  action  chosen  by  pilot 

Level  of  air  traffic  managers’  mental  workload 

Other  pilots’  levels  of  SA 

Air  traffic  managers’  levels  of  SA 

Level  of  confidence  in  planned  action 

Level  of  pilot  mental  workload 

Type  of  action  chosen  by  pilot 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  other  instruments 

Time  spent  reading  displays  in  the  system 

Level  of  confidence  in  perception  of  the  world 

Level  of  pilot  mental  workload 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  other  instruments 

Level  of  confidence  in  pilot’s  current  world  model 

Level  of  confidence  in  pilot’s  current  world  model 

Level  of  pilot  mental  workload 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  other  instruments 

Level  of  confidence  in  perception  of  the  world 

Perceived  time  pressure 

Ownship  state 

Type  of  action  chosen  by  pilot 

Current  team  SA 

Ownship  (immediate)  destination 

Level  of  confidence  in  automated  systems  performing  control 
actions 

Type  of  action  chosen  by  pilot 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  displays  in  the  system 

Time  spent  reading  other  instruments 

Ownship  (immediate)  destination 

Type  of  action  chosen  by  pilot 

Time  spent  reading  displays  in  the  system 

Perceived  time  pressure 

Level  of  confidence  in  current  machine  model 

Level  of  pilot  mental  workload 

Time  spent  viewing  out-the-window  information 

Time  spent  reading  displays  in  the  system 

Other  pilots’  levels  of  SA 

Level  of  air  traffic  managers’  mental  workload 

Other  pilots’  amount  of  mental  workload 

Air  traffic  managers’  levels  of  SA 

Level  of  air  traffic  managers’  mental  workload 

Other  pilots’  amount  of  mental  workload 

Accuracy  of  pilot’s  current  world  model  ★ 

Ownship  state 

Current  team  SA 

Level  of  confidence  in  perception  of  the  AMES  display 

Level  of  pilot  mental  workload 

Time  spent  reading  displays  in  the  system 

Level  of  confidence  in  position  sensors 

Ownship  state 

Time  spent  reading  displays  in  the  system 

Destination  (immediate)  of  other  aircraft 

Type  of  action  chosen  by  pilot 

Weather  ^ 

Wind 

Wind  “  T 

Weather 

Amount  of  out-the-window  information  available  to  the  pilot 

Ownship  state 

Amount  of  noise  in  cockpit  environment 

Current  team  SA 

Ownship  state 

Ownship  state 

Accuracy  in  predicting  the  machine’s  behavior 

Level  of  pilot  mental  workload 

Physical  state  of  pilot 

Ownship  state 
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Table  D5.  Matrix  Representing  the  Relations  Between  Only  the  Most  Important 
Characteristics. 
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Table  D6.  Scenarios  Representing  a  Combination  of  the  Most  Influential  and  Most  Sensitive  Characteristics. 

Driver/  Receiver 

1.  Omship  state /Level  of  air  traffic  managers *  mental  workload:.  For  obvious  reasons,  the  state  of  ownship  affects  the 
air  traffic  manager’s  amount  of  mental  workload.  For  example,  if  the  position  of  ownship  is  within  5  nm  of 
another  aircraft,  the  free-flight  environment  requires  the  air  traffic  manager  to  assume  responsibilities  that  are 
otherwise  the  pilot’s  responsibilities.  Therefore,  the  imagined  scenario  is  one  in  which  the  air  traffic  manager’s 
amount  of  mental  workload  is  high  because  ownship’s  alert  zone  contact  has  been  intruded. 

2.  Ownship  state  / Accuracy  of  pilot's  current  world  modeh  This  pairing  was  already  addressed  when  tradeoffs  were  ex¬ 
amined,  and  it  was  identified  as  having  a  direct  relation.  However,  what  is  currently  of  interest  is  the  influence 
of  ownship  state  on  the  accuracy  of  the  pilot’s  current  world  model.  As  stated  when  addressing  tradeoffs,  an 
undesirable  aircraft  attitude  might  affect  the  pilot’s  ability  to  obtain  information  from  the  world  (and  therefore 
negatively  affect  the  accuracy  of  the  world  model).  Therefore,  the  imagined  scenario  is  one  in  which  the  world 
model  is  inaccurate  because  ownship  state  does  not  allow  the  pilot  to  obtain  the  relevant  information  from  the 
world. 

3.  Ownship  state/ Accuracy  of  current  machine  modeh  These  two  characteristics  were  already  identified  as  having  a  trad¬ 
eoff  relation.  However,  what  is  currently  of  interest  is  only  the  influence  of  ownship  state  on  the  accuracy  of 
the  pilot’s  current  machine  model.  There  are  many  cases  in  which  ownship  state  would  have  a  negative  effect 
on  the  current  machine  model.  Specifically,  when  ownship  state  is  undesirable  in  any  manner  that  is  unrelated 
to  traffic  (e.g.,  altitude  is  too  low  given  the  weather  situation),  the  pilot  presumably  would  spend  more  time  at¬ 
tending  to  other  displays,  out-the-window-information,  and  control  devices.  Therefore,  the  imagined  scenario 
is  one  in  which  the  machine  model  is  inaccurate  because  the  pilot  is  attending  to  sources  other  than  the  CDTI. 

4.  Ownship  state/ Appropriateness  of  planned  action :  For  obvious  reasons,  the  state  of  ownship  has  an  effect  on 
whether  the  planned  action  is  appropriate.  For  example,  if  ownship  is  suddenly  in  an  undesirable  state  (e.g., 
on  a  course  or  at  a  speed  that  will  lead  to  an  alert  zone  contact),  the  action  that  is  planned  may  be  inappropri¬ 
ate  (e.g.,  the  planned  action  may  be  to  take  no  action).  Similarly,  if  ownship  is  in  a  desirable  state,  the  planned 
action  may  be  inappropriate  (e.g.,  changes  in  heading  create  an  alert  zone  contact).  Therefore,  the  imagined 
scenario  is  one  in  which  the  state  of  the  ownship  causes  the  planned  action  to  be  one  that  is  inappropriate. 

5.  Ownship  state/ Level  of  confidence  in  planned  action :  A  scenario  is  easily  imagined  in  which  ownship  state  influences 
the  pilot’s  level  of  confidence  in  the  planned  action.  Specifically,  if  the  position  of  ownship  is  such  that  an 
alert  zone  conflict  is  occurring,  the  pilot  is  required  to  make  a  decision  quickly.  Therefore,  it  might  be  difficult 
for  the  pilot  to  have  complete  confidence  in  such  a  decision. 

6.  Ownship  state /Level  of  pilot  mental  workload.  These  two  characteristics  were  already  identified  as  having  a  tradeoff 
relation.  While  the  inverse  relation  is  discussed  below,  what  is  currently  of  interest  is  only  the  influence  of 
ownship  state  on  the  level  of  pilot  mental  workload.  As  discussed  in  terms  of  tradeoffs,  an  example  of  this 
pairing  would  occur  if  aircraft  altitude  were  unacceptable  (i.e.,  the  current  altitude  will  create  an  alert  zone 
contact).  Pilot  mental  workload  problem  would  increase  in  the  attempt  to  rectify  the  problem  (e.g.,  attain  the 
correct  altitude  or  contact  the  appropriate  pilot). 

7.  Ownship  state /Type  of  action  chosen  ty  the  pilot  These  characteristics  were  already  identified  as  having  a  direct 
relation  in  the  section  that  addresses  tradeoffs.  As  indicated  in  that  section,  the  acceptability  of  ownship  state 
affects  the  action  chosen  by  the  pilot,  in  that  as  ownship  state  becomes  less  desirable  the  action  chosen  by  the 
pilot  may  be  less  desirable  (e.g,  in  extreme  cases,  the  pilot  may  panic  and  use  poor  judgment). 
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8.  Weather / Ownship  state:  For  obvious  reasons  (e.g.,  lightning,  wind),  the  weather  can  affect  the  state  of  ownship 
and  the  pilot's  ability  to  control  ownship, 

9.  Weather  /  Level  of  air  traffic  managers’  mental  workload \  Weather  affects  the  air  traffic  manager's  amount  of  work¬ 
load,  Under  normal  conditions,  the  air  traffic  manager  is  responsible  only  for  monitoring  the  ability  of  the 
pilots  to  maintain  self-separation.  With  the  advent  of  a  storm,  the  air  traffic  manager  must  also  monitor  the 
ability  of  the  pilots  to  avoid  weather  formations.  Under  severe  weather  conditions,  the  air  traffic  manager  pre¬ 
sumably  will  receive  more  verbal  reports  regarding  the  presence  and  severity  of  the  current  weather  situation. 

10.  Weather/ Accuracy  of  pilot’s  current  world  model  The  weather  may  affect  the  pilot's  current  world  model  in  cases 
where  it  limits  visibility.  Therefore,  the  pilot  may  lose  visual  contact  with  nearby  aircraft.  Because  the  pilot 
ma\  have  to  concentrate  on  controlling  the  aircraft,  it  indirectly  may  affect  the  model  bv  diverting  attention 
from  all  other  relevant  information  in  the  world  (traffic  in  this  case). 

11.  Weather/ Accuracy  of  current  machine  model  Similar  to  the  previous  effect,  weather  may  affect  the  machine  model 
by  diverting  attention  from  the  CDTI. 

12.  W eather/ Appropriateness  of  planned  action :  For  obvious  reasons,  the  surrounding  weather  situation  has  an  effect 
on  whether  the  planned  action  is  appropriate.  For  example,  under  normal  circumstances,  no  action  by  the 
pilot  may  be  appropriate.  However,  planning  the  same  lack  of  action  may  be  inappropriate  if  the  aircraft  is 
headed  for  a  weather  formation, 

13.  lb  father /Level  of  confidence  in  planned  action :  Poor  weather  may  affect  the  level  of  confidence  in  planned  action 
because  of  its  mere  existence  (i.e.,  the  pilot  may  wonder  whether  the  aircraft  will  avoid  the  weather  given  the 
current  plans).  Here,  what  is  more  important  is  the  effect  of  the  weather  on  the  traffic  situation.  Imagine 
ownship  is  in  congested  airspace  and  the  weather  is  poor.  Due  to  low7  visibility,  the  pilot  may  have  relatively 
little  confidence  that  the  planned  action  will  result  in  the  successful  avoidance  of  all  aircraft  alert  zones. 

14.  Weather/ Level  of  pilot  mental  workload:  Because  of  the  mere  existence  of  poor  weather,  the  pilot's  mental 
workload  increases.  Specifically,  the  pilot’s  attention  may  be  focused  upon  the  avoidance  of  the  weather  and 
controlling  the  aircraft  within  these  conditions.  Therefore,  in  poor  weather  conditions,  the  pilot’s  ability  to 
monitor  the  traffic  situation  (whether  through  the  world  or  the  CDTI)  may  be  hindered  by  the  limits  of 
working  memory. 

15.  Weather /Type  of  action  chosen  by  the  pilot  This  relation  is  similar  to  the  one  between  Weather  and  Appropriate¬ 
ness  of  Planned  Action,  For  obvious  reasons,  the  surrounding  w^eafher  situation  has  an  effect  on  whether  the 
control  action  is  appropriate.  For  example,  under  normal  circumstances,  no  action  by  the  pilot  may  be  appro¬ 
priate.  However,  under  poor  weather  conditions,  the  same  lack  of  action  may  place  the  aircraft  in  a  weather 
formation, 

16.  Time  of  day/ Ownship  state.  The  pilots  previous  actions  may  have  been  related  to  whether  the  aircraft  was  flying 
under  IFR  or  VFR  rules.  Therefore,  time  of  day  can  have  an  indirect  effect  on  the  acceptability  of  the  current 
state  of  ownship, 

17.  Time  of  day/  Level  of  air  traffic  managers’  mental  workload  Although  it  depends  on  the  sector,  the  volume  of  traf¬ 
fic  varies  with  the  time  of  day.  Therefore,  air  traffic  manager's  workload  varies  with  the  time  of  day, 

1 8.  Time  of  day/ Accuracy  of  pilot’s  current  world  model  Too  much  sunlight  and  lack  of  sunlight  can  affect  the  pilot's 
ability  to  view7  out-the-window  information.  Therefore,  the  accuracy  of  the  pilot's  current  world  model  can 
vary  with  the  time  of  day. 
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19.  Time  of  day/ Accuracy  of  current  machine  modeh  Glare  and  direct  sun  light  can  affect  the  pilot’s  ability  to  view  the 
CDTI.  Therefore,  the  accuracy  of  the  current  machine  model  can  vary  with  the  time  of  day. 

20.  Time  of  day /Level  of  confidence  in  planned  action :  Volume  of  traffic  and  visibility  both  vary  with  time  of  day.  Day¬ 
time  traffic  volumes  and  nighttime  low  visibility  conditions  will  both  decrease  the  pilot’s  confidence  that  alert 
zone  conflicts  will  be  avoided  successfully. 

21.  Time  of  day /Level  of  pilot  mental  Workload-Similar  to  the  previous  description,  daytime  traffic  volumes  and  night¬ 
time  low  visibility  conditions  will  both  increase  the  pilot’s  level  of  mental  workload.  As  a  result,  the  pilot’s 
attention  may  not  be  directed  at  the  most  relevant  information/ 

22.  Time  of  day /Type  of  action  chosen  by  the  pilot  The  action  chosen  by  the  pilot  may  vary  with  time  of  day.  Night¬ 
time  conditions  call  for  IRF  rules,  and  the  pilot  is  working  under  conditions  of  little  or  no  visibility.  Daytime 
conditions,  on  the  other  hand,  may  yield  high  traffic  volumes  and  strong  sunlight.  Given  these  hindrances,  the 
action  chosen  by  the  pilot  may  sometimes  be  inappropriate. 

23.  Terrain/ Omship  state.  Assuming  the  pilot  was  accurate,  the  flight  plans  take  the  terrain  into  account.  There¬ 
fore,  the  terrain  presumably  affects  ownship  state.  However,  if  the  pilot  fails  to  remember  the  geography,  a 
situation  could  arise  where  an  alert  zone  contact  is  unavoidable  (Le.,  two  planes  are  trapped  by  geographical 
boundaries). 

24.  Terrain/  Level  of  air  traffic  managers'  mental  workload f:  Presumably,  the  effects  of  terrain  would  vary  between  air 
traffic  managers  (rather  than  within  an  air  traffic  manager).  Specifically,  a  sector  with  dangerous  terrain  would 
be  more  difficult  to  monitor  than  a  sector  with  flat  terrain.  Therefore,  an  air  traffic  manager  assigned  to  a  sec¬ 
tor  with  dangerous  terrain  might  spend  relatively  more  time  assisting  aircraft  in  avoiding  terrain.  As  a  result, 
such  an  air  traffic  manager  might  be  less  effective  at  monitoring  the  self-separation  of  aircraft. 

25.  Terrain/ Accuracy  of  pilot's  current  world  modeh.  The  accuracy  of  the  pilot’s  current  world  model  might  be  sacri¬ 
ficed  if  terrain  occluded  out-the-window  information.  In  addition,  given  knowledge  of  perceptual  illusions, 
the  pilot  may  misjudge  distance  to  a  particular  geographical  formation  (e.g.,  a  mountain).  Misjudging  distance 
to  the  terrain  may  also  lead  to  misjudgments  regarding  the  position  of  visible  aircraft. 

26.  Terrain/ Appropriateness  of  planned  action :  For  obvious  reasons,  the  surrounding  terrain  has  an  effect  on  whether 
the  planned  action  is  appropriate.  For  example,  if  the  pilot  plans  an  action  that  deviates  from  the  original 
flight  plan,  the  newly  planned  action  may  be  appropriate.  However,  the  same  change  of  plans  may  result  in  a 
collision  with  terrain  or  a  situation  where  the  terrain  makes  an  alert  zone  contact  unavoidable  (i.e.,  two  aircraft 
are  trapped).  . 

27.  Terrain/  Level  of  confidence  in  planned  action :  Because  terrain  can  occlude  out-the-window  information,  the  pilot 
may  have  relatively  litde  confidence  that  the  planned  action  will  result  in  the  successful  avoidance  of  alert 
zones. 

28.  Terrain/ Level  of  pilot  mental  workload.  When  an  aircraft  nears  potentially  dangerous  terrain,  the  pilot  is  more 
likely  to  utilize  attentional  resources  on  the  avoidance  of  the  terrain.  Therefore,  the  pilot  is  more  likely  to 
ignore  traffic  information  from  both  the  CDTI  and  the  forward  field  of  view. 

29.  Terrain /Type  of  action  chosen  by  the  pilot  The  pilot  may  suddenly  change  the  state  of  the  aircraft  if  the  aircraft 
nears  terrain.  As  a  result,  the  pilot  unknowingly  may  place  the  aircraft  in  an  alert  zone  conflict  or  an  alert  zone 
conflict  watch/warning. 
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30,  Perceived  time  pressure /  Ownship  state :  A  pilot  under  time  pressures  is  more  likely  to  ignore  certain  aspects  of  the 
environment.  Therefore,  the  acceptability  of  ownship  state  may  suffer  (e.g.,  ownship  may  be  on  a  collision 
course), 

31 ,  Perceived  time  pressure j Accuracy  of  pikfs  current  world  model:  If  a  crisis  arises  and  it  is  not  related  to  self-separa¬ 
tion,  the  pilot  obviously  will  spend  relatively  less  time  attending  to  the  traffic-related  information.  As  a  result, 
the  out-the-window  information  regarding  traffic  or  radio  communications  regarding  traffic  may  be  ignored. 
Therefore,  the  risk  of  a  separation  violation  increases, 

32,  Perceived  time  pressure  j  Accuracy  of  current  machine  model.  If  a  crisis  arises  and  it  is  not  related  to  self-separation, 
the  pilot  obviously  will  spend  relatively  less  time  attending  to  the  CDTL  As  a  result,  the  pilot’s  machine  model 
will  be  less  accurate,  and  the  risk  of  a  separation  violation  will  be  more  likely. 

33,  Perceived  time  pressure/  Appropriateness  of  planned  action :  If  the  pilot  feels  pressured  by  time,  the  pilot  may  make 
decisions  too  quickly.  As  a  result,  the  planned  action  is  less  likely  to  be  appropriate, 

34,  Perceived  time  pressure  j  Level  of  confidence  in  planned  action:.  If  the  pilot  feels  pressured  by  time,  the  pilot  may  make 
decisions  too  quickly.  As  a  result,  the  pilot  may  feel  less  than  confident  in  the  action  that  has  been  planned, 

35,  Perceived  time  pressure /Level  of  pilot  mental  workload*.  When  a  pilot  feels  the  pressures  of  time,  working  memory  is 
taxed.  As  a  result,  mental  workload  necessarily  increases.  If  mental  workload  is  too  high,  the  pilot  is  not  able 
to  attend  to  all  relevant  stimuli  in  the  environment.  Therefore,  if  the  situation  that  requires  attention  is  not 
related  to  self-separation,  information  regarding  the  traffic  situation  may  be  what  is  ignored. 

36,  Perceived  time  pressure /Type  of  action  chosen  by  the  pilot.  Although  this  pair  has  already  been  identified  as  represent¬ 
ing  a  tradeoff,  what  is  of  interest  here  is  the  influence  that  perceived  time  pressure  has  on  the  action  chosen 
by  the  pilot.  If  the  pilot  feels  pressured  by  time,  the  pilot  may  make  decisions  too  quickly.  As  a  result,  the  cho¬ 
sen  action  may  be  less  than  satisfactory  (i.e,,  the  aircraft  may  come  into  an  alert  zone  contact). 

37,  Level  of  pilot  mental  workload!  Ownship  state:  These  two  characteristics  were  already  identified  as  having  a  trad¬ 
eoff  relation,  What  is  important  here  is  the  effect  mental  workload  has  on  ownship  state.  As  pilot  mental 
workload  increases,  the  pilot  is  more  likely  to  ignore  some  responsibilities.  Therefore,  as  workload  increases,  it 
is  possible  for  the  acceptability  of  ownship  state  to  decrease. 

38,  Level  of  pilot  mental  workload!  Level  of  air  traffic  managers*  mental  workload.  These  characteristics  have  already 
been  identified  as  having  an  inverse  relation.  Specifically,  if  the  pilot  performed  most  of  the  duties  associated 
with  self-separation,  the  pilot’s  mental  workload  would  increase  in  high-density  traffic  situations.  The  fewer 
self-separation  responsibilities  the  pilot  performed,  the  higher  the  ATM’s  mental  workload  would  be.  These 
characteristics  were  also  shown  to  have  a  direct  relation.  For  example,  a  pilot  under  conditions  of  high  mental 
workload  may  not  respond  immediately  to  an  ATM.  As  a  result,  the  ATM’s  mental  workload  may  be  increased 
by  the  pilot’s  high  mental  workload, 

39,  Level  of  pilot  menial  workload! Accuracy  of  pilofs  current  world  model.  As  pilot  mental  workload  increases,  the  pilot 
is  more  likely  to  ignore  some  relevant  information  in  the  world  (e.g.,  radio  communications).  Therefore,  as 
workload  increases,  it  is  possible  that  the  pilot  will  have  a  less  accurate  “picture”  of  traffic-related  informa¬ 
tion, 

40,  Level  of  pilot  mental  workload) Accuracy  of  current  machine  model.  As  pilot  mental  workload  increases,  the  pilot  is 
more  likely  to  ignore  some  relevant  information.  Therefore,  as  workload  increases,  it  is  possible  that  the  pilot 
will  attend  less  to  the  CDTI, 
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41.  Level  of  pilot  mental  workload/ Appropriateness  of  planned  action'.  As  pilot  mental  workload  increases,  the  pilot  is 
more  likely  to  ignore  some  relevant  information  in  the  world.  Therefore,  as  workload  increases,  it  is  possible 
that  the  pilot  will  make  hastier  decisions  and  possibly  plan  a  dangerous  course  of  action  (e.g.,  one  that  leads 
to  a  penetration  of  the  alert  zone). 

42.  Level  of  pilot  mental  workload /Level  of  confidence  in  planned  action :  These  characteristics  have  already  been  identified 
as  representing  a  tradeoff  relation.  However,  what  is  important  here  is  the  effect  of  workload  on  the  pilot’s 
confidence  in  the  planned  action.  High  pilot  workload  might  decrease  the  pilot’s  confidence  in  the  planned 
action.  Under  high  workload  conditions,  time  may  not  allow  the  pilot  to  assess  all  traffic-related  information 
and  leave  the  pilot  feeling  less  than  confident. 

43.  Level  of  pilot  mental  workload /Type  of  action  chosen  by  the  pilot  This  pairing  has  already  been  identified  as  repre¬ 
senting  a  tradeoff.  However,  what  is  of  interest  are  the  effects  of  workload  on  the  type  of  action  chosen. 

As  pilot  mental  workload  increases,  the  pilot  is  more  likely  to  overlook  relevant  information.  Therefore,  as 
workload  increases,  it  is  possible  for  the  quality  of  the  chosen  action  to  decrease. 

44.  General piloting  skills/ Ownship  state'.  Because  a  pilot’s  experience,  ability,  and  training  affect  decision  making,  the 
state  of  the  ownship  (acceptable  vs.  unacceptable)  will,  on  average,  correlate  with  the  pilot’s  level  of  skill. 

45.  General piloting  skills/  Level  of  air  traffic  managers'  mental  workload  As  a  pilot’s  experience,  ability,  and  training 
increase,  the  amount  of  assistance  needed  by  the  ATM  presumably  decreases.  Therefore,  a  pilot  with  a  great 
amount  of  skill  will  lessen  the  mental  workload  of  the  ATM. 

46.  General piloting  skills/  Accuracy  of  pilot's  current  world  model.  A  pilot  with  a  great  amount  of  skill  is  generally  better 
at  knowing  when  their  “picture”  of  the  situation  is  incomplete.  Therefore,  a  pilot  with  a  great  amount  of  skill 
will  know  when  to  attend  to  information  from  the  forward  field  of  view  or  radio  communications. 

47.  General piloting  skills /  Accuracy  of  current  machine  model  A  pilot  with  a  great  amount  of  skill  is  generally  better  at 
knowing  when  their  “picture”  of  the  situation  is  incomplete.  Therefore,  a  pilot  with  a  great  amount  of  skill 
will  know  when  to  attend  to  information  on  the  CDTI. 

48.  General piloting  skills / Appropriateness  of  planned  action'.  A  pilot  with  a  great  amount  of  skill  is  generally  more  able 
to  predict  what  will  happen  in  the  future.  Therefore,  the  skilled  pilot,  on  the  average,  will  generally  plan  an 
action  that  is  more  appropriate. 

49.  General piloting  skills/  Level  of  confidence  in  planned  action :  A  pilot  with  more  skill  will  be  more  confident  in  plan¬ 
ning  their  actions.  A  pilot  with  less  skills  will  feel  less  confident  in  their  ability  and  experience,  and  as  a  result, 
confidence  will  be  relatively  lower. 

50.  General piloting  skills /Level  of  pilot  mental  workload.  A  pilot  with  less  skill  presumable  is  not  able  to  “chunk”  in¬ 
formation  as  well  as  the  more  experienced  pilot.  Therefore,  put  in  the  same  situation,  pilots  of  different  skill 
levels  may  experience  different  amounts  of  mental  workload. 

51.  General piloting  skills/  Type  of  action  chosen  by  the  pilot  The  pilot  with  greater  skills  will  presumably  choose  actions 
that  may  not  be  the  most  desirable,  but  they  achieve  the  objective  and  are  efficient.  The  pilot  with  less  skill 
may,  on  average,  choose  a  less  appropriate  action  than  that  of  the  pilot  with  great  skills. 
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Table  D7.  Information  Requirements  for  Traffic  Awareness  in  the  Free-Flight  Environment 

Note:  Highlighted  information  requirements  are  identical  to  those  presented  by  either  Endsely  et  aL  (1998)  or 
Endsley  and  Rodgers  (1994), 

*  =  knowledge  >  =  information 


Pilot: 

•  Ownship  state 

>  Identification  of  ownship 

^  Horizontal  position  of  ownship 

>  Heading  of  ownship 
^  Speed  of  ownship 

>  Vertical  speed  of  ownship 

>  lAltitude  of  ownship 
Attitude  of  ownship 

y  Immediate  destination  of  ownship 
y  Route  of  ownship 

•  Future  of  ownship  state 

>  Future  horizontal  position  of  ownship 
^  Future  heading  of  ownship 

^  Future  speed  of  ownship 
y  Future  vertical  speed  of  ownship 
^  Future  altitude  of  ownship 
^  Future  destination  of  ownship 

>  Future  route  of  ownship 

^  Future  violations  of  aircraft  capabilities 
(e.g.,  speed  restrictions) 

•  Potential  for  future  alert  zone  contact 

>  Alert  zone  contact  watch 
^  Alert  zone  contact  warning 

•  Ownship  planned  changes 

^  Heading  changes  of  ownship 
^  Speed  changes  of  ownship 
^  Altitude  changes  of  ownship 
^  Immediate  destination  changes  of  own- 
ship 

^  Route  changes  of  ownship 

•  Hypothetical  results  of  planned  changes 

y  Hypothetical  horizontal  position  of  own- 
ship 

^  Hypothetical  heading  of  ownship 
y  Hypothetical  speed  of  ownship 
y  Hypothetical  vertical  speed  of  ownship 


^  Hypothetical  altitude  of  ownship 

>  Hypothetical  immediate  destination  of 
ownship 

y  Hypothetical  route  of  ownship 

>  Hypothetical  violations  of  aircraft  capa¬ 
bilities  (e.g.,  speed  restrictions) 

^  Hypothetical  alert  zone  watch 
y  Hypothetical  alert  zone  warning 
5^  Hypothetical  alert  zone  contact 

Other  aircraft  states 
^  Id  of  other  aircraft 
y  p^smoii  of  other  aircraft 

^  Heading  of  other  aircraft 
y  Speed  of  other  aircraft 
^  Vertical  speed  of  other  aircraft 
y  Altitude  of  other  aircraft 
^  Immediate  destination  of  other  aircraft 

>  Route  of  other  aircraft 

Future  of  other  aircraft  states 

>  Future  horizontal  position  of  other  air¬ 
craft 

^  Future  heading  of  other  aircraft 
y  Future  speed  of  other  aircraft 

>  Future  vertical  speed  of  other  aircraft 
y  Future  altitude  of  other  aircraft 

>  Future  destination  of  other  aircraft 

>  Future  route  of  other  aircraft 

Number  of  aircraft  nearby 

y  Number  of  aircraft  that  are  within  a  user- 
specified  range  of  ownship  (This  range 
may  vary  depending  on  the  day,  the  loca¬ 
tion,  or  the  situation,) 

Occurrence  of  alert  zone  warning/watch 

>  Auditory  signal  of  an  alert  zone  warning/ 
watch 

>  Type  of  alert  zone  warning/watch  (tem¬ 
porary  or  one  that  is  likely  to  lead  to  alert 
zone  contact) 
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•  Best  plan  of  action  to  avoid  a  future  or 
discontinue  a  current  alert  zone  contact 

>  Optimal  changes  to  horizontal  position  of 
ownship 

^  Optimal  changes  to  heading  of  ownship 
y  Optimal  changes  to  speed  of  ownship 
y  Optimal  changes  to  vertical  speed  of 
ownship 

^  Optimal  changes  to  altitude  of  ownship 

>  Optimal  changes  to  destination  of  own- 
ship 

^  Optimal  changes  to  route  of  ownship 

>  Request  other  pilot  to  make  changes 

•  Weather  formation  existence 

y  Ownship  within  weather  formation 
(Inform  pilot  to  consult  ATM  or  display 
outside  of  system) 

y  Ownship  headed  for  weather  formation 
(Inform  pilot  to  consult  ATM  or  display 
outside  of  system) 

•  Existence  of  Potentially  Dangerous  Ter¬ 
rain 

y  Ownship  headed  for  potentially  danger¬ 
ous  terrain  (Inform  pilot  to  consult  ATM 
or  display  outside  of  system) 

•  Time  frame  allowed  for  ignoring  the  CDTI 

y  Time  until  the  next  alert  zone  contact 
watch/warning 

•  Current  CDTI  settings 

>  Current  brightness  settings 
^  Current  contrast  settings 

ATM: 

•  Aircraft  states 
y  Id  of  aircraft 

y  Horizontal  position  of  aircraft 

>  Heading  of  aircraft 
^  Speed  of  aircraft 

y  Vertical  speed  of  aircraft 
^  Altitude  of  aircraft 
y  Immediate  destination  of  aircraft 
^  Route  of  aircraft 


•  Future  of  aircraft  state 

^  Future  horizontal  position  of  aircraft 

>  Future  heading  of  aircraft 

>  Future  speed  of  aircraft 

y  Future  vertical  speed  of  aircraft 
^  Future  altitude  of  aircraft 

>  Future  destination  of  aircraft 

>  Future  route  of  aircraft 

•  Aircraft  Changes 

y  Heading  changes  of  aircraft 
y  Speed  changes  of  aircraft 
y  Altitude  changes  of  aircraft 
y  Immediate  destination  changes  of  aircraft 
y  Route  changes  of  aircraft 

•  Weather  formation  avoidance 

y  Aircraft  within  weather  formation 
^  Aircraft  headed  for  weather  formation 
^  Optimal  changes  to  altitudes  of  aircraft 
^  Optimal  changes  to  routes  of  aircraft 

•  Terrain  avoidance 

y  Aircraft  headed  for  terrain 
y  Optimal  changes  to  altitudes  of  aircraft 
y  Optimal  changes  to  routes  of  aircraft 

•  Time  frame  allowed  for  working  with  the 
present  conflict 

y  Time  until  the  next  alert  zone  contact 
watch/warning 

>  Number  of  aircraft  pairs  that  will  have  an 
,  alert  zone  contact  watch/warning  within 

a  user-specified  span  of  time  (This  range 
may  vary  depending  on  the  day,  the  sec¬ 
tor,  or  the  situation.) 

•  Potential  for  future  alert  zone  contact 

y  Alert  zone  contact  watches 
y  Alert  zone  contact  warnings 

•  Pilots  in  need  of  assistance 

>  Aircraft  requesting  immediate  assistance 
^  Aircraft  requesting  long-term  assistance 
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